10.  TextFinder Project

TextFinder walks a directory tree and reports every file whose content matches a user-supplied regular expression. Five implementations - Rust (baseline and optimized), C++, C#, and Python - share the same three-component architecture and command-line interface, making the project a controlled cross-language comparison.

Why TextFinder is useful for language comparison

1. The workload is real: filesystem traversal, regex matching, and console output are idioms used in production tools, not toy examples.
2. All five variants produce identical output, so correctness is easy to verify by diffing results against a reference run.
3. Performance differences are measurable and explainable - the bottleneck is OS I/O, the regex engine choice matters, and startup cost is visible in short-running processes.
4. The spec-driven workflow (Constitution.md, Structure.md, Spec.md) was used to build each variant, so you can compare how the same spec translates into different language idioms.

10.1  Architecture

Each implementation is a three-component library wired together by an entry point. No library depends on another library; all cross-component communication flows through the entry point using callback delegates (C#, Python) or a trait (Rust).

CommandLine   DirNav   Output
      \          |       /
           EntryPoint

• CommandLine - parses /Key [Value] tokens from argv; exposes typed option values to callers.
• DirNav - depth-first directory walk; fires callbacks on each directory and file entered.
• Output (also named TextFinder in the Rust variants) - performs the regex match and writes matching paths to the console.
• EntryPoint - wires the three components together; no direct cross-dependencies between them.

The Rust variant expresses the DirNav callback contract as a trait. EntryPoint provides a concrete type (TfAppl) that implements it:

impl dir_nav_lib::DirEvent for TfAppl {
    fn do_dir(&mut self, d: &str) {
        self.curr_dir = d.to_string();
        if !self.get_hide() { print!("\n--{}", d); }
    }
    fn do_file(&mut self, f: &str) {
        let fqf = format!("{}/{}", self.curr_dir, f);
        if self.tf.find(&fqf) {
            if self.tf.get_last_path() != self.curr_dir && self.get_hide() {
                print!("\n\n  {}", self.curr_dir);
                self.tf.last_path(&self.curr_dir);
            }
            print!("\n      {:?}", f);
        }
    }
}

10.2  Command-Line Interface

All five implementations accept the same flags. Both / and - prefixes work - use - in Git Bash to avoid path conversion.

Option	Meaning	Default
/P <path>	Root directory to search	. (current directory)
/p <ext,...>	File extensions to include (comma-separated)	all files
/r <regex>	Regular expression matched against file content	. (any)
/s	Recurse into subdirectories	true
/H	true: show only directories with matches. false: show every directory entered.	true
/v	Verbose - echo all options before searching	off
/h	Print help and exit	off

Example - find .cs files containing Action starting from the current directory:

CsTextFinder /P . /p cs /r "Action"

10.3  Skip Lists

DirNav in every variant maintains a hard-coded skip list of directory names that are never entered during traversal. Entries are matched against the bare directory name, so a bin folder at any depth is skipped regardless of where it appears.

Category	Skipped directory names
Build artifacts	bin, obj (C#); target (Rust); build, out (C++)
Python	__pycache__, .venv, venv, dist
VCS / IDE	.git, .vs, .idea
Archives	archive

10.4  Language Implementations

Project	Language	Build tool	Entry point
CppTextFinder	C++23 (named modules)	CMake 3.28+ / MSVC or Clang	build/EntryPoint/Release/text_finder
CsTextFinder	C# / .NET 10	dotnet CLI	CsTextFinder.exe
PyTextFinder	Python 3.10+	none (run directly)	python EntryPoint/PyTextFinder.py
rs_textfinder	Rust (Cargo workspace)	cargo	cargo run from EntryPoint/
rs_textfinder_opt	Rust (Cargo workspace)	cargo	cargo run from EntryPoint/

Each project also carries a generate_part.py script that calls the Claude API to regenerate any component from its Spec.md:

python generate_part.py CommandLine
python generate_part.py DirNav
python generate_part.py Output
python generate_part.py EntryPoint

10.5  Performance

Timings over 20 warm-cache runs (first discarded), searching the NewSite root for class across source files. Median = average of the 10th and 11th sorted values.

TextFinder	Files Visited	Files Matched	Min (s)	Median (s)	Max (s)
PyTextFinder	1196	656	0.222	0.281	0.715
EntryPointOpt	1196	656	0.536	0.610	1.034
CppTextFinder	1196	656	0.568	0.647	0.706
CsTextFinder	1196	656	0.827	1.053	1.456
EntryPoint	1196	656	0.873	0.905	1.402

All five agree on 656 matched files, confirming behavioral equivalence. The elevated max values for EntryPointOpt, CsTextFinder, and EntryPoint reflect OS scheduling interrupts during a run, not intrinsic tool cost - the medians are more representative of steady-state performance.

Why Python leads despite being interpreted

• The workload is I/O-bound. Every implementation spends most of its time in OS calls - readdir, open, read - which cost the same in every language because they all wait on the same kernel.
• Python’s hot path is C. os.scandir(), file.read(), and re.search() are all C extensions; Python only interprets the thin control-flow glue between them.
• C++ std::regex is slow. It uses a backtracking NFA engine that rescans input on every match attempt. Python’s re and Rust’s regex crate both use DFA-based engines that scan in a single linear pass.
• C# pays .NET startup cost. The runtime and JIT spin up before the first directory is touched, consuming a large fraction of total elapsed time for this short-running tool.

10.6  Optimization Story

rs_textfinder_opt investigates three changes to the baseline Rust implementation. Only the third produced a clear gain. Step 1 - pre-compile the regex (no gain). The baseline compiles the pattern inside find(), once per file. Storing a compiled Option<regex::bytes::Regex> and building it once in the regex() setter eliminated that work, but elapsed time was unchanged - the regex crate’s DFA construction is fast enough that it does not dominate per-file cost. Step 2 - search raw bytes (minor gain). The baseline uses read_to_string (UTF-8 validation + heap allocation) with a lossy-UTF-8 fallback for binary files. Switching to regex::bytes::Regex and reading every file as raw &[u8] eliminated the double-allocation fallback path. The gain was small because most files are valid UTF-8 on the first attempt. Step 3 - use DirEntry::file_type() instead of Path::is_dir() (dominant gain). The baseline calls path.is_dir() inside the directory scan loop, issuing a separate stat syscall per entry. DirEntry::file_type() returns the type the OS already cached as part of the readdir response - no extra syscall. This one change reduced median elapsed time from ~0.91 s to ~0.61 s, a 33% improvement, and accounts for nearly all of the gain between the two Rust variants.

// baseline -- issues a stat syscall for every directory entry
if entry.path().is_dir() { ... }

// optimized -- type is cached from the readdir response; no extra syscall
if entry.file_type()?.is_dir() { ... }

10.7  Code Metrics

Generated by code_metrics.py from the Projects directory. Lines = total line count (code + comments + blanks). Scopes = scope-opening tokens: { count for brace languages; lines ending with : for Python.

CppTextFinder

File	Lines	Scopes
CommandLine\src\CmdLine.ixx	126	29
CommandLine\src\test.cpp	225	79
DirNav\src\DirNav.ixx	126	19
DirNav\src\test.cpp	491	82
EntryPoint\src\main.cpp	67	8
EntryPoint\src\test.cpp	435	59
generate_part.py	339	28
Output\src\Output.ixx	110	17
Output\src\test.cpp	469	70
TOTAL	2388	391

CsTextFinder

File	Lines	Scopes
CommandLine\CmdLine.cs	91	11
CommandLine\Test.cs	50	12
DirNav\DirNav.cs	87	16
DirNav\Test.cs	184	47
EntryPoint\Program.cs	57	14
EntryPoint\Test.cs	138	35
generate_part.py	159	14
Output\Output.cs	66	20
Output\Test.cs	166	37
TOTAL	998	206

PyTextFinder

File	Lines	Scopes
CommandLine\cmd_line.py	90	20
CommandLine\test_cmd_line.py	70	20
DirNav\dir_nav.py	80	23
DirNav\test_dir_nav.py	153	35
EntryPoint\PyTextFinder.py	70	9
EntryPoint\test_main.py	115	35
generate_part.py	156	12
Output\output.py	57	20
Output\test_output.py	114	33
TOTAL	905	207

EntryPoint

File	Lines	Scopes
RustCmdLine\examples\test1.rs	43	15
RustCmdLine\src\cmd_line_lib.rs	242	52
RustDirNav\examples\test1.rs	77	14
RustDirNav\src\dir_nav_lib.rs	294	53
EntryPoint\src\text_finder.rs	297	77
RustTfVerify\src\main.rs	760	137
TOTAL	1715	348

EntryPointOpt

File	Lines	Scopes
RustCmdLine\examples\test1.rs	43	15
RustCmdLine\src\cmd_line_lib.rs	243	52
RustDirNav\examples\test1.rs	77	14
RustDirNav\src\dir_nav_lib.rs	292	52
EntryPoint\src\text_finder.rs	319	78
RustTfVerify\src\main.rs	760	137
TOTAL	1736	348

10.8  References

Resource	Description
Code Story: Spec-Driven Development	The workflow used to generate each TextFinder component from its Spec.md.
Code Story: Experiment Arena	Timing tools, profilers, and how to interpret the performance numbers.
Rust regex crate	DFA-based regex engine used in both Rust TextFinder variants.