Spec-Driven

Spec-Driven Rust: Dirnav

traversal, admission, matching, and emission in one generic struct

Synopsis:
This page covers rust_textfinder_dirnav, which walks, selects, reads, matches, and formats. It is the only component that touches file contents, and it writes to no stream.
  • A struct generic over the Output trait it declares and does not implement, so the polymorphism is a compile-time bound rather than a runtime dispatch.
  • Two shared borrows and one unique borrow are the whole of the access control this component needs, and it is checked rather than documented.
  • Nothing in traversal panics - every fallible call yields an io::Result and every one is matched. No unwrap, no expect.
Four standard-library calls are rejected by name, each because the plausible call is subtly wrong.
  • fs::metadata follows a link and would hide the case the specification requires be announced.
  • Path::extension returns nothing for .gitignore.
  • str::lines implements two of the three terminators, so a classic Mac OS file would arrive as one line.
  • A directory-walking crate would own the descent this library must own.
Two details are worth reading for what the language decides rather than the specification.
  • str::from_utf8 supplies all four UTF-8 rejections, where C++ writes them out by hand for want of such a call.
  • The file count is incremented at two call sites rather than inside examine - the one place this implementation's shape forced a choice the other two did not face.
  • The hand-written Lines iterator is what lets a search stop reading at the first match.

1.  Public Interface

rust_textfinder_dirnav walks a directory tree, reads each selected file, evaluates the expression against each line, and formats every matching file into the block Spec_TextFinder.md §3.4 fixes, emitting each of its lines as it is produced. It is the only component that touches file contents, and it writes to no stream. The crate root exports three things.
pub trait Output {
    fn output(&mut self, text: &str);
}

pub type SkipList = Vec<String>;

pub struct Dirnav<'a, O: Output> { /* private */ }

impl<'a, O: Output> Dirnav<'a, O> {
    pub fn new(out: &'a mut O, skips: &'a SkipList, commands: &'a ProgramCommands)
        -> Result<Self, regex::Error>;

    pub fn search(&mut self, root: &Path);

    pub fn emit_run_summary(&mut self);
}
The struct declares the Output trait it does not implement, which is what puts rust_textfinder_output downstream of it in the dependency chain. The generic parameter makes the call direct, so the polymorphism is a compile-time bound rather than a runtime dispatch. new compiles commands.regex_text with regex::Regex::new - the engine Spec_TextFinder.md §6.1 assigns to Rust - and returns Err carrying the crate's own error when the expression will not compile. The failure is a returned value rather than a panic, and the specification says why in one sentence: a malformed pattern is something the user typed, so it is an ordinary outcome of construction. rust_textfinder_cmdline guarantees the text is non-empty, so new never sees an empty expression. All three arguments are borrowed for the lifetime 'a and are owned by rust_textfinder_entry, which the borrow checker obliges to keep them alive for the lifetime of the Dirnav. The skip list and the commands are shared borrows and cannot be modified through them; out is a unique borrow, since emitting a line mutates the sink. That one distinction is the whole of the access control this component needs, and it is checked rather than documented. search returns nothing. Every failure it meets is announced through Output, so the caller has nothing to report on its behalf. A single value is reused across every root path, so the compiled expression is built once per run, and search carries no state from one call to the next but for the two run counts of Section 5.1, which accumulate across calls by design. emit_run_summary writes the run summary Spec_TextFinder.md §3.6 requires. It takes &mut self because emitting mutates the sink, takes no other argument, and returns nothing; rust_textfinder_entry calls it once after the last search returns. The split is deliberate: the counts belong here because nothing else sees the entries, and the call belongs to the caller because nothing here knows which root was the last. No accessor is exposed for either count - the line is the whole of what they are for, and the unit suite reads them by reading that line through its own Output.

2.  Traversal

search resolves a root before walking it. A symbolic link, a kind that is neither regular file nor directory, and a failed metadata query each draw cannot open; a regular file is examined as a single file; a directory is walked. The skip list is never consulted for the root, which Spec_TextFinder.md §3.2 exempts because the user named it explicitly. The root's kind comes from std::fs::symlink_metadata and not std::fs::metadata. The specification rejects the second by name: it follows a symbolic link and would report the target's kind, hiding the very case §3.2 requires be announced. The recursion is written explicitly. walk iterates one directory and calls itself on each subdirectory it decides to enter, so entries are handled as std::fs::read_dir yields them, without being collected or reordered. Two choices are stated in the specification rather than left to the implementer.
  • No directory-walking crate. walkdir and its kind own the descent this library must own, since /s and the skip list both decide whether to enter a directory, and such a crate may impose an order of its own.
  • read_dir enumerates a single level only. Its order is unspecified by the standard library and is whatever readdir or FindFirstFileW returns, which is the platform facility §3.2 requires an implementation to enumerate through. Entries are neither sorted nor grouped, as §3.2 forbids.
Every fallible call in the walk yields an io::Result and every one of them is matched. read_dir failing announces the directory and returns; an entry yielded as Err announces the directory and returns, so a directory that becomes unreadable part way through is announced rather than silently truncated; a failed file_type announces that entry and continues with the next. No unwrap and no expect appear, which is what the specification means when it says nothing in traversal panics. The order of the entry tests matters, and §5 fixes it.
  1. Symbolic link, tested first. An entry that is a link is passed over silently whatever its target, since no attempt is made to open it. The kind comes from DirEntry::file_type, which does not follow a link; std::fs::metadata is rejected by name here for the same reason it is rejected for the root.
  2. Unrenderable name, tested second. An entry whose name is not valid UTF-8 draws cannot open and is neither entered nor searched. OsStr::to_str reports the condition by returning None, so nothing propagates and nothing is caught.
  3. Skip list, extension filter, and the open, after both.
Putting the name test before the skip-list test is what makes such a name announced whatever /p holds, and Spec_TextFinder.md §3.4 writes that rule over every such file rather than over the selected ones. This position is a correction: the Rust text originally let the extension filter reach the entry first, which would have made the same tree and the same command line produce different stdout from the two implementations. The Process page covers how that was found. The link test staying ahead of the name test is a deliberate asymmetry all three implementations carry: a link whose name cannot be rendered stays silent, because rule 1 turns on the entry's kind, which costs no decoding, and §3.2 asks for silence there. A pruned directory draws no announcement. Pruning is work the library chose not to do, and cannot open reports work it could not do.

3.  File Selection

Selection follows the /p rules of Spec_TextFinder.md §5, and the list arrives from rust_textfinder_cmdline already normalized to bare extensions. An empty list selects every file; a non-empty list selects a file whose extension matches an entry. std::path::Path::extension does not implement those rules, and the specification rejects it by name: it returns None for .gitignore, whereas §5 gives that file the extension gitignore. The extension is therefore taken as the text after the last . in the file name, with rfind('.') and a slice, no special case for a leading dot, and a name holding no . at all has no extension. Selection never meets a name it cannot read. The Section 2 gate has already refused every entry whose name is not valid UTF-8, so selected takes a &str rather than an OsStr and has no failure case of its own. Neither an empty nor a non-empty /p list changes what happens to such a file. Comparison goes through same_name, which is case-sensitive on POSIX and case-insensitive on Windows per §3.2 and §5. It is written twice, once under #[cfg(windows)] and once under #[cfg(not(windows))], and it is the only place in the file where the target platform changes behavior. The same function serves the skip list, since §3.2 and §5 fix one comparison for both, and the specification names the function rather than restating the rule so that the two cannot drift. The Windows comparison uses eq_ignore_ascii_case, so it folds ASCII only. An extension holding a non-ASCII character therefore compares case-sensitively on both platforms, which the specification states as a consequence rather than leaving a reader to discover it. Selection applies uniformly: a root path that is a regular file is filtered by /p like any other file. A root named on /P escapes the skip list and does not escape the extension filter.

4.  Admission and the No-Content Case

examine applies the three admission tests of Spec_TextFinder.md §3.3 in the order the specification fixes. The size test reads the length std::fs::Metadata::len reports, so a file above the limit is never read into memory; the NUL and UTF-8 tests read bytes. The limit is 10,485,760 bytes, the number §3.3 fixes, written 10_485_760 so that a reader can count the digits. new records once, for the whole run, whether the no-content case applies:
let path_line_only =
    commands.regex_text == "." && !commands.line_numbers && !commands.matched_line;
When it holds, a selected file that passes the size test and is not empty produces a block of its path line alone, and the file is never opened. No file announcement accompanies that block: searched reports a file that was read and matched nothing and skipped reports one a content test rejected, and in this case neither happened, so the library emits neither whatever /h says. A selected file of zero size produces no block and draws no announcement either. Both error announcements still work in this case, because both rest on metadata: a file above the limit draws too large, and one whose metadata cannot be read draws cannot open. A file that needs reading is read in full with std::fs::read, so one failing a later test is skipped entirely rather than searched in part. The NUL test is bytes.contains(&0). The UTF-8 test is std::str::from_utf8, which rejects truncated sequences, overlong encodings, encoded surrogates, and scalar values above U+10FFFF - the four rejections §3.3 requires - so this implementation performs no validation of its own and the standard library is the authority for what valid UTF-8 means. The C++ implementation writes those four rejections out by hand, one line each, because it has no such call to lean on. A leading UTF-8 BOM is stripped with strip_prefix('\u{FEFF}') after the admission tests, not before, so its three bytes count toward the size limit and toward the NUL scan like any others.

5.  Matching and Emission

Each line is evaluated with Regex::is_match, which gives the anywhere-in-the-line match §3.3 requires and asks the engine for nothing more - no match position, no matched substring, no capture group. The line is passed as &str, so ., a character class, and a class escape each match one Unicode scalar value. §6.1 records std::regex over char as the one engine of the four that matches a byte instead, so on a line holding a non-ASCII character this implementation agrees with the C# and Python implementations and the C++ one is the outlier. Three rules govern the emission, and all three are visible in a dozen lines of examine:
  1. The path line goes out at the first match, ahead of the detail line for that same match, and a path_written flag makes sure it goes out once. A file that never matches produces no line at all; a file that matches many produces its path line once.
  2. With neither /n nor /L, the loop returns as soon as the path line is written. The block has no detail lines, so the first match settles the file.
  3. Otherwise the loop runs to the end of the file, writing one detail line - two spaces of indent, then the fields /n and /L select - as each matching line is evaluated.
Nothing is accumulated for the file. Output receives each line as it is produced, which is the pipeline behavior §3.4's "as they occur" requires. Announcements go through the same Output, at the point the failure or the finding is met, and an error announcement is not gated on /h. The /h gate is one function, file_announcement, and it reads as the rule it implements: emit unless suppress_on_no_match. A file announcement reports only a file that produced no block, so it never repeats a path the output already carries - searched once the last line has been evaluated, which is the first moment the library knows the file matched nothing, and skipped at the point of rejection. Path rendering is where this library does its own string work rather than the standard library's. §3.4 requires / on every platform for the whole of <path>, so a path is built by joining the root's own text with the entry names descended through, and Path::display is rejected by name: it renders the platform's separator, which on Windows would emit \ and make the same tree produce different output on two platforms. The root's own text is normalized too, since §3.4's rule covers all of <path>. normalize replaces every \ with / before the first entry name is appended, so -P src\sub on Windows yields src/sub/file.rs and not src\sub/file.rs. That was a gap the audit found, and the integration suite now asserts it. A root path of . contributes no leading ./, which walk arranges by taking an empty prefix for that root and join by returning the name alone when the prefix is empty. One announcement carries a path that is not the path on disk. An entry name that is not valid UTF-8 cannot be rendered as text, and §3.4 fixes the outcome: the file is not searched and draws cannot open, naming it with U+FFFD substituted for each unit that will not render. OsStr::to_string_lossy does exactly that, so this library needs no substitution code. Every name that survives the Section 2 gate renders without substitution, so a block's path line is always the name itself. That is the one place the two implementations solve the same requirement with different amounts of code. to_string_lossy is the whole of it here; C++ writes a renderPath helper that walks bytes on POSIX and UTF-16 units on Windows, because the two calls it would otherwise use behave differently by platform and one of them throws.

5.1  The Run Summary

Spec_TextFinder.md §3.6 puts the two run counts in this library, for every implementation alike, and fixes the line they produce. Two usize fields hold them, both zero from new, and neither is reset by search, so they accumulate over every root the value is given.
  • Directories are counted at the head of walk, before fs::read_dir, so a directory that cannot be enumerated is counted and announced alike.
  • Files are counted where the /p test of Section 3 admits one, at both of the two places that test is applied - the root-path arm of search and the file arm of walk - and ahead of the metadata call in each.
Putting the file increment at the two call sites rather than inside examine is the one place this implementation's shape forced a choice the other two did not face. walk announces cannot open and never calls examine when entry.metadata() fails, and that file passed /p, so an increment inside examine would miss it and this implementation would report one file fewer than the others over the same tree. The specification says so in its own §8.1, because the next reader's instinct will be to fold the two increments into one. Everything §3.6 excludes is excluded by where those increments sit rather than by a test of its own: an entry refused by /p, a symbolic link, an entry beneath a pruned directory, an entry whose name is not valid UTF-8, and an entry that is neither a file nor a directory all fail or bypass the /p test - though the last two draw cannot open on the way past, which is why the file count can be smaller than the number of announcements a run writes. emit_run_summary composes the line with format! and sends it through the same Output as every other line, in §3.6's fixed form and with neither noun inflected:
accessed <files> files, <directories> directories
It is not gated on /h, which governs file announcements alone, and it names no path, so this section's rendering rules do not reach it. Eight unit tests cover it, and the one worth naming asserts accessed 1 files, 0 directories from a root that is a regular file - the uninflected singular, asserted rather than left to be noticed.

6.  Line Splitting Without str::lines

Spec_TextFinder.md §3.3 makes LF, CRLF, and a bare CR each terminate a line, and treats a final unterminated run as a line. str::lines is rejected by name because it implements two of those three: it splits on LF and strips a trailing CR, and it does not treat a bare CR as a terminator, so a classic Mac OS file would arrive as one line. The library carries a Lines iterator of its own, 30 lines including the Iterator impl. It holds Option<&str> for the remainder, finds the next \r or \n, consumes two bytes when the tail starts with \r\n and one otherwise, and yields None when the remainder is empty. Starting from None for empty input is what makes an empty file produce no line rather than one empty line. Writing it as an Iterator rather than as a function returning Vec<&str> is what lets rule 2 of Section 5 stop reading at the first match: the caller's for loop returns and the remaining bytes are never scanned. Each item borrows the buffer rather than copying it, so a 10 MB file costs one allocation for its contents and nothing per line. enumerate supplies the line number, offset by one at the point of use. Line numbers count every line, matching or not, which follows from numbering the iterator rather than the matches.

7.  Source

lib.rs in full - 261 lines, the largest file in the project. The comments cite the Rust specification's sections, which cite Spec_TextFinder.md in turn.
Rust_Spec_driven_Dirnav/src/lib.rs
//! rust_textfinder_dirnav - directory navigation, matching, and block formatting.
//! Implements Spec_Rust_TextFinder_Dirnav.md, the Rust binding of Spec_TextFinder.md sections 3.2-3.4.

use regex::Regex;
use rust_textfinder_cmdline::ProgramCommands;
use std::fs::{self, Metadata};
use std::path::Path;

#[cfg(test)]
mod unit_tests;

pub trait Output {
    fn output(&mut self, text: &str);
}

pub type SkipList = Vec<String>;

pub struct Dirnav<'a, O: Output> {
    out: &'a mut O,
    skips: &'a SkipList,
    commands: &'a ProgramCommands,
    expression: Regex,
    path_line_only: bool,
    files: usize,
    directories: usize,
}

const SIZE_LIMIT: u64 = 10_485_760;

impl<'a, O: Output> Dirnav<'a, O> {
    pub fn new(
        out: &'a mut O,
        skips: &'a SkipList,
        commands: &'a ProgramCommands,
    ) -> Result<Self, regex::Error> {
        let expression = Regex::new(&commands.regex_text)?;
        let path_line_only =
            commands.regex_text == "." && !commands.line_numbers && !commands.matched_line;
        Ok(Dirnav { out, skips, commands, expression, path_line_only, files: 0, directories: 0 })
    }

    /// Section 8.1: the run summary of Spec_TextFinder.md section 3.6, written once
    /// after the last root.
    pub fn emit_run_summary(&mut self) {
        let text = format!("accessed {} files, {} directories", self.files, self.directories);
        self.out.output(&text);
    }

    pub fn search(&mut self, root: &Path) {
        let display = normalize(&root.to_string_lossy());
        let info = match fs::symlink_metadata(root) {
            Ok(info) => info,
            Err(_) => return self.announce("cannot open", &display),
        };
        let kind = info.file_type();
        if kind.is_symlink() || !(kind.is_file() || kind.is_dir()) {
            self.announce("cannot open", &display);
        } else if kind.is_file() {
            if self.selected(basename(&display)) {
                self.files += 1;   // section 8.1: after the /p test, ahead of every later outcome
                self.examine(root, &display, &info);
            }
        } else {
            self.walk(root, &display);
        }
    }

    fn walk(&mut self, dir: &Path, display: &str) {
        self.directories += 1;   // section 8.1: counted before read_dir, so one that fails counts too
        let prefix = if display == "." { "" } else { display };
        let entries = match fs::read_dir(dir) {
            Ok(entries) => entries,
            Err(_) => return self.announce("cannot open", display),
        };
        for entry in entries {
            let entry = match entry {
                Ok(entry) => entry,
                Err(_) => return self.announce("cannot open", display),
            };
            let raw = entry.file_name();
            let kind = match entry.file_type() {
                Ok(kind) => kind,
                Err(_) => {
                    let child = join(prefix, &raw.to_string_lossy());
                    self.announce("cannot open", &child);
                    continue;
                }
            };
            if kind.is_symlink() {
                continue;
            }
            let name = match raw.to_str() {
                Some(name) => name,
                None => {
                    let child = join(prefix, &raw.to_string_lossy());
                    self.announce("cannot open", &child);
                    continue;
                }
            };
            let child = join(prefix, name);
            if kind.is_dir() {
                if self.commands.recurse && !self.skips.iter().any(|skip| same_name(skip, name)) {
                    self.walk(&entry.path(), &child);
                }
            } else if kind.is_file() {
                if self.selected(name) {
                    self.files += 1;   // section 8.1
                    match entry.metadata() {
                        Ok(info) => self.examine(&entry.path(), &child, &info),
                        Err(_) => self.announce("cannot open", &child),
                    }
                }
            } else {
                self.announce("cannot open", &child);
            }
        }
    }

    fn examine(&mut self, path: &Path, display: &str, info: &Metadata) {
        if info.len() > SIZE_LIMIT {
            return self.announce("too large", display);
        }
        if self.path_line_only {
            if info.len() > 0 {
                self.out.output(display);
            }
            return;
        }
        let bytes = match fs::read(path) {
            Ok(bytes) => bytes,
            Err(_) => return self.announce("cannot open", display),
        };
        if bytes.contains(&0) {
            return self.file_announcement("skipped", display);
        }
        let text = match std::str::from_utf8(&bytes) {
            Ok(text) => text,
            Err(_) => return self.file_announcement("skipped", display),
        };
        let text = text.strip_prefix('\u{FEFF}').unwrap_or(text);

        let details = self.commands.line_numbers || self.commands.matched_line;
        let mut path_written = false;
        for (number, line) in Lines::over(text).enumerate() {
            if !self.expression.is_match(line) {
                continue;
            }
            if !path_written {
                self.out.output(display);
                path_written = true;
            }
            if !details {
                return;
            }
            let detail = self.detail(number + 1, line);
            self.out.output(&detail);
        }
        if !path_written {
            self.file_announcement("searched", display);
        }
    }

    fn detail(&self, number: usize, line: &str) -> String {
        let mut text = String::from("  ");
        if self.commands.line_numbers {
            text.push_str(&number.to_string());
            if self.commands.matched_line {
                text.push_str(" - ");
            }
        }
        if self.commands.matched_line {
            text.push_str(line);
        }
        text
    }

    fn selected(&self, name: &str) -> bool {
        if self.commands.extensions.is_empty() {
            return true;
        }
        match name.rfind('.') {
            None => false,
            Some(dot) => {
                let extension = &name[dot + 1..];
                self.commands.extensions.iter().any(|candidate| same_name(candidate, extension))
            }
        }
    }

    fn announce(&mut self, kind: &str, path: &str) {
        self.out.output(&format!("{kind} {path}"));
    }

    fn file_announcement(&mut self, kind: &str, path: &str) {
        if !self.commands.suppress_on_no_match {
            self.announce(kind, path);
        }
    }
}

/// Spec_TextFinder.md section 3.4: every path is rendered with `/` on every platform.
fn normalize(text: &str) -> String {
    text.replace('\\', "/")
}

fn join(prefix: &str, name: &str) -> String {
    if prefix.is_empty() {
        String::from(name)
    } else if prefix.ends_with('/') {
        format!("{prefix}{name}")
    } else {
        format!("{prefix}/{name}")
    }
}

fn basename(display: &str) -> &str {
    display.rsplit('/').next().unwrap_or(display)
}

#[cfg(windows)]
fn same_name(left: &str, right: &str) -> bool {
    left.eq_ignore_ascii_case(right)
}

#[cfg(not(windows))]
fn same_name(left: &str, right: &str) -> bool {
    left == right
}

/// Line splitting per Spec_TextFinder.md section 3.3: LF, CRLF, and bare CR terminate
/// a line, and a final unterminated run is a line. `str::lines` does neither.
struct Lines<'t> {
    rest: Option<&'t str>,
}

impl<'t> Lines<'t> {
    fn over(text: &'t str) -> Self {
        Lines { rest: if text.is_empty() { None } else { Some(text) } }
    }
}

impl<'t> Iterator for Lines<'t> {
    type Item = &'t str;

    fn next(&mut self) -> Option<&'t str> {
        let rest = self.rest?;
        match rest.find(|c| c == '\r' || c == '\n') {
            None => {
                self.rest = None;
                Some(rest)
            }
            Some(at) => {
                let (line, tail) = rest.split_at(at);
                let consumed = if tail.starts_with("\r\n") { 2 } else { 1 };
                let remainder = &tail[consumed..];
                self.rest = if remainder.is_empty() { None } else { Some(remainder) };
                Some(line)
            }
        }
    }
}
Rust_Spec_driven_Dirnav/Cargo.toml
[package]
name = "rust_textfinder_dirnav"
version = "0.1.0"
edition = "2021"
rust-version = "1.70"

[lib]
path = "src/lib.rs"

[dependencies]
regex = "1"
rust_textfinder_cmdline = { path = "../Rust_Spec_driven_Cmdline" }
This manifest is the only one in the project that names a third-party crate. It does not name rust_textfinder_output, which supplies the generic argument at the point of use rather than at the point of definition, and that absence is the dependency direction of Section 2 of the Structure page stated in a build file.

8.  Prompt Records

This page carries none. Page_Structure.md §8 assigns it Prompts_Spec_Rust_TextFinder_Dirnav.md and its Fix companion, and neither was written: the Dirnav specification was produced in the same turn as the other four documents, recorded at the project level. Three audit items reach this component - the unnormalized root separators, the unnamed skip-list folding rule, and the gate's position - and all three sit on the Process page.