Spec-Driven

Spec-Driven C#: Testing

four suites, 172 checks, and no dotnet test

Synopsis:
This page covers the four suites and what they leave uncovered. Three unit suites for the three libraries, one integration suite driving the built executable.
  • 172 checks, each one a named claim. Comparable with the C++ page's 151 assertions and not with the Rust page's 88 tests.
  • Suites sort emitted lines before comparing wherever more than one directory entry is involved, since the specification leaves that order to the filesystem.
The dependency rule bars a third-party framework, which rules out the way C# is normally tested.
  • Every test adapter arrives as a NuGet package, so dotnet test is unavailable and each suite is a console project returning its failure count.
  • Rust is the one of the three that gets a runner from its language - the same rule costs Rust nothing and costs C# a harness.
  • Separate projects mean a suite sees only its component's public surface, and no InternalsVisibleTo is used.
Every check passes, and Section 6 names four claims that are read rather than checked.
  • The unrenderable name - the rule that cost two implementations a code change is tested in none of the three.
  • The failed write, and exit code 2, neither of which a run of the executable can provoke.
  • The POSIX side of everything, which a .NET assembly makes most visible, the same binary taking the other branch on a second machine without being rebuilt.

1.  Four Suites

Three unit suites cover the three libraries and one integration suite covers the binary by driving the built executable end to end. The division follows from what CSharp_TextFinder_Entry is: a binary whose behavior is its startup sequence, its exit codes, and its stream routing, none of which can be exercised without running it. Spec_TextFinder.md §6.2 asks every implementation for unit suites one per library component, one integration suite driving the built executable, one demonstration, and a runner per kind that announces each suite and exits with the number that failed. A suite that was never built counts as a failure rather than passing by absence.
Suite Checks What it covers
Cmdline 67 Every default, both introducers, boolean case folding, last-occurrence-wins, /P accumulation, the six trimmed characters and the one not trimmed, all six parse diagnostics, the 21-line help text, and the nine listing lines
Dirnav 31 Skip-list pruning including the root carve-out, /p selection with the dot-file rule, /s false, all four block forms, the /h gating, cannot open, LF, CRLF and bare CR splitting, BOM consumption, and the 10 MB limit at and just above the boundary
Output 7 The one-sink rule, the release of that right on disposal, idempotent disposal, a write after disposal, both write methods, and dispatch through the interface and through a generic constraint
Integration 56 The executable end to end: exit codes, stream routing, /H, the bare command line, the /v ordering, separator normalization, the absence of a BOM, and all seven usage diagnostics
172 checks in total, each one a named claim. The counts are comparable with the C++ suites' 151 assertions and not with the Rust suites' 88 tests, since a Rust #[test] holds as many assertions as its claim needs and a check here is one assertion with one name. Two suites deserve a note on what they prove beyond their counts. The Dirnav suite builds a temp tree and supplies its own Recorder, a class holding one List<string> that implements IOutput, which satisfies the same constraint the real sink does and so shows that the generic parameter binds to any implementation of the interface. The Output suite cannot read stdout back, since the sink owns the only writer over it, so it checks what does not depend on reading the stream and leaves the bytes to Section 5. The Dirnav and integration suites sort emitted lines before comparing wherever more than one directory entry is involved, because §3.2 leaves the order of a directory's entries to the filesystem. A suite that compared unsorted output would be asserting a property the specification declines to guarantee.

2.  The Harness, and Why Not dotnet test

§6.2 bars a third-party test framework, and that one sentence rules out the way C# is normally tested. dotnet test discovers and runs tests through a test adapter, and every adapter - xUnit, NUnit, MSTest - arrives as a NuGet package. The dependency rule of §6 admits a package only for regex and filesystem access, and the framework supplies both. Each suite is therefore a console project whose Main runs its checks and returns the number that failed, and the runners invoke those executables directly. That is the arrangement the C++ implementation reaches for the same reason, its *_TestDriver.cpp mapping a failure count to an exit status. Rust is the one of the three that gets a runner from its language: #[test] and cargo test ship with it, so §6's confinement costs Rust nothing and costs C# a harness. The harness is a Check class of about 30 lines.
internal sealed class Check
{
    private int _failures;
    private int _total;

    public void That(bool ok, string name) => Record(ok, name);

    public void Equal(string actual, string expected, string name)
    {
        bool ok = actual == expected;
        Record(ok, name);
        if (ok) return;
        Console.WriteLine("          expected: [" + Visible(expected) + "]");
        Console.WriteLine("          actual:   [" + Visible(actual) + "]");
    }

    // The CRLF failures of a first run look identical to their expected text without this.
    private static string Visible(string text) =>
        text.Replace("\r", "\\r").Replace("\n", "\\n");
}
Visible earns its place from the other implementations' experience rather than from this one's. The C++ integration suite's first run failed eight assertions whose expected and actual text looked identical, which is the signature of a CRLF difference, and that suite gained the same helper afterward. Here it was written before the first run and has never yet printed anything. The Check class is duplicated across the four suites rather than shared. The C++ project made that choice deliberately - a shared support module was proposed, would have removed about seventy lines, and was rejected so that each component could be read and built alone. This thread follows it, and in C# the cost is lower than it looks: a shared harness would be a fifth project that every suite references, and the duplication is 30 lines against one more node in the dependency graph. Each unit suite sits in its component's test/ folder as its own project, which is what §6.2's "beside the code it tests" asks for and what costs each library project one <Compile Remove="test/**" /> item. The Structure page covers that trade. One consequence of separate projects is worth naming: a unit suite sees only its component's public surface. Rust's suites reach private items through #[cfg(test)] mod unit_tests; and use super::*;, and the C# suites cannot without an InternalsVisibleTo attribute. None is used. Every claim §3 through §5 fixes is observable through the public API, so the suites test behavior rather than internals, and the one place that hurts is the Output suite, which cannot inspect the private failed state and asserts only that no write throws. The integration suite locates the executable from its own assembly's directory, walking up to the folder CSharp_TextFinder_Structure.md names and back down to the binary's output path, with a command-line argument overriding it. C++ gets that string from a CMake generator expression and Rust from env!("CARGO_BIN_EXE_rust_textfinder") at compile time; the .NET SDK offers no equivalent for a project that references the binary without referencing its assembly, so the path is computed from the layout the structure document fixes. Both the Dirnav and integration suites build temp trees through a TempTree whose Dispose removes the directory, so a failing check leaves nothing behind. Each tree's name carries the process id and an incrementing counter.

3.  The Runners

Three batch files sit at the top of the C# folder. Each builds the solution first, brackets every suite with a === starting ... banner and a === ... returned N line, then prints a summary and returns the number of failed suites as its own exit code.
Runner What it does
run_unit_tests.bat Runs the Cmdline, Dirnav, and Output suite executables in turn
run_integration_tests.bat Runs the integration suite; an argument overrides the executable under test
run_demo.bat Runs the demonstration and supplies the capture date through TEXTFINDER_DEMO_DATE
Each takes its working directory from its own location with cd /d "%~dp0", so any of the three runs from any working directory. Each builds the solution before its first suite, which is what §6.2 asks of a runner so that a fresh checkout needs no separate step and a reader cannot run yesterday's binary against today's source. dotnet build is incremental, so a run after a build costs about a second and one after an edit rebuilds only the projects that changed. The --verbosity quiet flag keeps the build's own output from burying the suite report. Two failure paths follow from that, and §6.2 requires both. A build that fails stops the runner before any suite and counts every suite it would have run as failed. And a suite executable that is absent after a build that succeeded prints not built: with its path and counts as a failure - which catches a renamed project or a changed output path, the cases a green run would otherwise hide. Each also holds the console after its summary, so a reader who starts one from a file manager sees its output rather than a window that closes. The hold is one :hold subroutine, it comes last, and it does not change the exit code - a pause leaves ERRORLEVEL untouched, and the runner returns its failure count explicitly in any case. Setting TEXTFINDER_NO_PAUSE to a non-empty value suppresses it, which is how the captures in Sections 4 and 5 were taken and how the demonstration of §7.5 is recaptured. run_demo.bat passes the local date in rather than letting the demonstration compute it. The demonstration falls back to the UTC date when the variable is absent, which reads as the previous day late in the evening, so the runner is the authority for the date a capture carries.

4.  Unit Suite Output

Captured 2026-09-16 from run_unit_tests.bat. Each suite names every check, prints its count, and ends with the line its own Main writes. The check names are the specification restated as claims, which is what makes a passing run readable as coverage rather than only as a number.
=== starting unit suite: CSharp_TextFinder_Cmdline_UnitTest
CSharp_TextFinder_Cmdline unit tests
  PASS  default /P is .
  PASS  default /p is empty
  PASS  default /r is .
  PASS  default /s is true
  PASS  default /h is true
  PASS  default /v is false
  PASS  default /H is false
  PASS  default /n is false
  PASS  default /L is false
  PASS  a new ProgramCommands equals the result of parsing an empty array
  PASS  the first element is a switch, not a program name
  PASS  a program name in args[0] is refused, since Main's array carries none
  PASS  introducers / and - are equivalent
  PASS  boolean values fold case
  PASS  /h and /H are distinct switches
  PASS  last occurrence wins for /r
  PASS  an argument beginning with an introducer is taken verbatim
  PASS  /P accumulates in the order given
  PASS  the first /P replaces the default even when equal to it
  PASS  items are split, trimmed, and stripped of one dot, and empties discarded
  PASS  only one leading dot is stripped
  PASS  duplicates are retained
  PASS  an all-separator list is empty
  PASS  extension case is not folded by the parser
  PASS  the six characters §5 names are trimmed
  PASS  no-break space is not trimmed, since §5 does not name it
  PASS  not a switch
  PASS  unrecognized switch
  PASS  an over-long token is unrecognized
  PASS  a bare introducer is unrecognized
  PASS  missing argument
  PASS  invalid boolean
  PASS  empty root path
  PASS  empty expression
  PASS  parsing stops at the first violation
  PASS  every diagnostic ends with the usage line
  PASS  a diagnostic carries LF and no CR
  PASS  the help text begins with the usage line
  PASS  the usage line names the executable
  PASS  the help text ends with a newline
  PASS  the help text carries LF and no CR
  PASS  the help text is the 22 lines §5.1 fixes
  PASS  help names the run summary of §3.6
  PASS  help lists /P
  PASS  help lists /p
  PASS  help lists /r
  PASS  help lists /s
  PASS  help lists /h
  PASS  help lists /v
  PASS  help lists /H
  PASS  help lists /n
  PASS  help lists /L
  PASS  the usage line is one line
  PASS  the listing of the defaults is the nine lines of §5.3, with /v false
  PASS  the listing of a bare -v true is the nine lines §5.3 gives
  PASS  one /P line per root path, in traversal order
  PASS  the extension list is joined by comma and space
  PASS  booleans render lower case, so bool.ToString is not used
  PASS  no line of the listing ends in whitespace: [/P .]
  PASS  no line of the listing ends in whitespace: [/p]
  PASS  no line of the listing ends in whitespace: [/r a b]
  PASS  no line of the listing ends in whitespace: [/s true]
  PASS  no line of the listing ends in whitespace: [/h true]
  PASS  no line of the listing ends in whitespace: [/v false]
  PASS  no line of the listing ends in whitespace: [/H false]
  PASS  no line of the listing ends in whitespace: [/n false]
  PASS  no line of the listing ends in whitespace: [/L false]
  PASS  /r renders the expression verbatim, backslash included
  68 of 68 passed
PASS
=== unit suite CSharp_TextFinder_Cmdline_UnitTest returned 0

=== starting unit suite: CSharp_TextFinder_Dirnav_UnitTest
CSharp_TextFinder_Dirnav unit tests
  PASS  a malformed expression throws RegexParseException from the constructor
  PASS  a skip-list directory is pruned silently
  PASS  a root named in the skip list is traversed, and pruning resumes below it
  PASS  /s false searches the root's own files and enters no subdirectory
  PASS  an empty /p list selects every file, files without an extension included
  PASS  a non-empty /p list selects by last dot suffix and excludes a file without one
  PASS  a dot-file's extension is its last dot-suffix
  PASS  a root path that is a regular file is searched
  PASS  a root path that is a regular file is filtered by /p like any other
  PASS  the default command line reports every non-empty file without opening it
  PASS  the default /h leaves only the matching file's block
  PASS  /h false announces every examined file exactly once
  PASS  a file above the limit draws an error announcement under /h true
  PASS  a file exactly at the limit is searched
  PASS  LF terminates a line
  PASS  CRLF terminates a line and the CR is not part of it
  PASS  a bare CR terminates a line
  PASS  a final unterminated run is a line
  PASS  a leading BOM is not part of the first line
  PASS  /n and /L true give a path line and a number-and-text detail line per match
  PASS  /L false leaves the number alone on the detail line
  PASS  /n false leaves the text alone on the detail line
  PASS  with neither /n nor /L a block is its path line alone
  PASS  line numbers count every matching line in line order
  PASS  a matching line yields one detail line however many occurrences it holds
  PASS  every path is rendered with forward separators
  PASS  every path begins with the root it was reached through
  PASS  a separator the user typed in a root path is normalized
  PASS  a root path that cannot be opened is announced
  PASS  an error announcement is not gated on /h
  PASS  one navigator serves every root path, in the order given
  PASS  every examined file and every entered directory is counted
  PASS  a file the /p list excluded is not counted
  PASS  a directory holding no selected file is still counted
  PASS  a pruned directory and its files are counted once the list no longer prunes it
  PASS  under /s false no subdirectory is counted
  PASS  a root that is a regular file counts as a file, and neither noun is inflected
  PASS  a root that cannot be opened is counted as neither
  PASS  the counts are of the whole run, not of one root
  PASS  an entry reached under two roots counts once for each
  PASS  the summary is not gated on /h
  41 of 41 passed
PASS
=== unit suite CSharp_TextFinder_Dirnav_UnitTest returned 0

=== starting unit suite: CSharp_TextFinder_Output_UnitTest
CSharp_TextFinder_Output unit tests
  PASS  a second sink is refused while the first lives
  PASS  the sink is the IOutput implementation Dirnav binds to
verbatim text, already terminated
one line, terminator added by the sink

  PASS  writing through both methods reports nothing and throws nothing
  PASS  a sink disposed releases the right to make one
  PASS  disposing twice is safe
  PASS  the interface method is reached through a generic constraint
reached through the constraint
  PASS  a write after disposal is discarded and reports nothing
  7 of 7 passed
PASS
=== unit suite CSharp_TextFinder_Output_UnitTest returned 0

0 unit suite(s) failed
Four groups of the 67 Cmdline names carry the C#-specific decisions rather than the parent specification's rules. the first element is a switch, not a program name and a program name in args[0] is refused, since Main's array carries none are the index-0 scan asserted from both sides. the six characters §5 names are trimmed and no-break space is not trimmed, since §5 does not name it bound the trim set in both directions, which is what rules out string.Trim(). only one leading dot is stripped rules out string.TrimStart('.'). And booleans render lower case, so bool.ToString is not used names the method it excludes. The nine no line of the listing ends in whitespace checks are one loop over the listing's lines rather than nine written claims, and each prints the line it examined. That is the rule §5.3 states as a property of the whole listing, asserted per line so that a failure names which one. The Dirnav suite tests the accepted costs rather than only the rules. the default command line reports every non-empty file without opening it is §3.3's no-content case with both of its costs in one claim: a binary file is reported, and an empty one is not. a file exactly at the limit is searched pairs with the line above it to pin 10,485,760 as a boundary rather than an approximation. And a separator the user typed in a root path is normalized is §3.4's rule applied to the one part of a path that is not an entry name. Twelve of its 41 checks cover the run summary of §3.6, and they read as that section's exclusion list turned into claims: a file the /p list excluded is not counted, a pruned directory and its files are counted once the list no longer prunes it, under /s false no subdirectory is counted, a root that cannot be opened is counted as neither. Two of the twelve test properties no other check could reach - that the counts are of the whole run rather than of one root, and that an entry reached under two roots counts once for each - by driving one navigator over two roots the way the binary does. And a root that is a regular file counts as a file, and neither noun is inflected asserts accessed 1 files, 0 directories verbatim, pinning the uninflected singular §3.6 accepts rather than leaving a later reader to correct it. Four lines of the Output capture are not the harness speaking. verbatim text, already terminated, one line, terminator added by the sink, the empty line after it, and reached through the constraint are what two of the checks wrote to the real stdout, which is the one thing this suite cannot redirect. The empty line is the Output("") case, and seeing it in the capture is the assertion made visible: an empty string still terminates a line. Those four lines appear at a different point in the list from run to run, and the reason is the component under test. The suite writes its own report to stderr precisely because the type it tests owns stdout, and when a runner sends both streams to one file the order is the operating system's rather than the program's - the sink's buffered lines arriving whenever its Dispose flushed them. A reader comparing two captures of this suite should expect those four lines to move and the seven PASS lines not to.

5.  Integration Suite Output

Captured in the same run, from run_integration_tests.bat. Each of these 56 checks spawns the executable one or more times, which is why this suite accounts for nearly all the runtime while the three in-process suites finish in hundredths of a second combined. Four of the 56 arrived with Spec_TextFinder.md §3.6, and three of them assert an absence: the bare command line, /H, and a malformed /r must each write no closing line, which is the only way to test a rule about runs that do not traverse. The fourth asserts that a traversing run's last line is its summary. The other 14 checks §3.6 touched were already here and simply gained the line in their expected stdout - which is what asserting exact output costs when the output grows, and what it buys.
=== starting integration suite: integration
CSharp_TextFinder integration tests
  PASS  a bare command line lists every default, with /v false
  PASS  a bare command line writes nothing to stderr
  PASS  a bare command line exits 0
  PASS  /H prints the text §5.1 fixes
  PASS  /H exits 0
  PASS  /H writes nothing to stderr
  PASS  the help text carries LF only
  PASS  stdout carries no byte-order mark
  PASS  /H is taken before any traversal
  PASS  /v lists the resolved options before the search output
  PASS  the search output follows the /v listing
  PASS  the run summary closes the output, per §3.6
  PASS  a bare command line traverses nothing and writes no run summary
  PASS  /H traverses nothing and writes no run summary
  PASS  a malformed expression traverses nothing and writes no run summary
  PASS  not a switch: P diagnostic
  PASS  not a switch: P exits 1
  PASS  not a switch: P leaves stdout empty
  PASS  unrecognized switch: /x diagnostic
  PASS  unrecognized switch: /x exits 1
  PASS  unrecognized switch: /x leaves stdout empty
  PASS  missing argument for switch: /P diagnostic
  PASS  missing argument for switch: /P exits 1
  PASS  missing argument for switch: /P leaves stdout empty
  PASS  invalid boolean for -s: yes diagnostic
  PASS  invalid boolean for -s: yes exits 1
  PASS  invalid boolean for -s: yes leaves stdout empty
  PASS  empty root path for switch: /P diagnostic
  PASS  empty root path for switch: /P exits 1
  PASS  empty root path for switch: /P leaves stdout empty
  PASS  empty expression for switch: /r diagnostic
  PASS  empty expression for switch: /r exits 1
  PASS  empty expression for switch: /r leaves stdout empty
  PASS  a malformed expression puts the option listing on stdout first
  PASS  the malformed-expression diagnostic follows on stderr
  PASS  a malformed expression exits 1
  PASS  under /v the listing is written once, not twice
  PASS  a block carries its path once and its detail lines beneath it
  PASS  a normal search exits 0
  PASS  a normal search writes nothing to stderr
  PASS  a root of . contributes no leading ./
  PASS  a named root is part of every path and its separators are normalized
  PASS  roots are traversed in the order given
  PASS  reversing the roots reverses the output
  PASS  the compiled skip list prunes a matching directory
  PASS  /s false enters no subdirectory
  PASS  /p filters by extension after normalization
  PASS  /h false announces every examined file exactly once
  PASS  the default /h hides only the files that matched nothing
  PASS  an unopenable root is announced through the output component
  PASS  an unopenable root does not affect the exit code
  PASS  an unopenable root writes nothing to stderr
  PASS  the no-content case reports a binary file and omits an empty one
  PASS  stdout carries no CR byte on any platform
  PASS  stdout ends with a terminator
  PASS  a CRLF file yields lines free of the carriage return
  56 of 56 passed
PASS
=== integration suite integration returned 0

0 integration suite(s) failed
The 18 checks whose names begin with a reason line are the six parse diagnostics tested three ways each - the diagnostic text, the exit code, and that stdout stayed empty - through one expect_usage_failure-style helper and one loop. The third of those three is the one a reader might not expect, and it is where the specification is strictest: six of the seven refusals leave stdout empty and the seventh puts the option listing there first, so testing only the text and the code would miss half the rule. Two checks read the bytes rather than the characters, and they are the ones this thread exists to make. stdout carries no CR byte on any platform scans the raw standard output for 0x0D, and stdout carries no byte-order mark checks the first two bytes are not 0xEF 0xBB. Both would pass trivially in Rust and both would have failed in C# had the sink used Console.WriteLine and Encoding.UTF8, which is the pair the Output specification rejects by name. a CRLF file yields lines free of the carriage return is the pair to the first of those: one asserts the program adds no CR, the other that it removes one the file carried. Together they cover §3.3's line splitting and §3.4's terminator from opposite ends. The suite compares expected stdout against CommandLine.HelpText() and CommandLine.UsageLine() imported from the library rather than against string literals of its own, so the fixture and the source cannot drift. It then checks two properties a literal would have caught anyway - that the help text begins with usage: CSharp_TextFinder [ and that it is 22 lines - so a library returning the wrong text consistently still fails.

6.  What the Suites Do Not Reach

Every check passes, and four claims in the specifications are unchecked. Naming them is worth more than the count above, since §6.2 asks the suites to check a claim rather than let it be read.
  • The unrenderable name. §3.4 settles what happens to a file whose name the implementation cannot render as text, and this implementation answers that the case cannot arise on Windows and resolves through a failed attributes query on POSIX. No fixture tests either half: building such a name means an unpaired surrogate on Windows, which a string carries without complaint, or a non-UTF-8 byte sequence on POSIX, which this machine cannot produce. The C++ and Rust suites report the same gap, so the rule that cost two implementations a code change is tested in none.
  • The failed write. Nothing breaks stdout mid-run, so the output failed notice, the permanence of the failed state, and the rule that one notice is written rather than one per discarded line are all read rather than checked.
  • Exit code 2 from the binary. The unit suite confirms that a second StdoutSink throws, but no run of the executable can provoke it, since the binary builds one sink. The other path, standard output failing to open, is not reachable from a test either.
  • The POSIX side of everything. The captures above are Windows runs. The comparison field folds case under OperatingSystem.IsWindows() and compares ordinally otherwise, and only the first branch is exercised here. This is the gap a .NET assembly makes most visible, since the same binary would take the other branch on a second machine without being rebuilt.
Three of the four are properties of failure paths, which is the ordinary shape of a coverage gap. The fourth is a platform, and it is the one a second machine would close without a line of new test code. One more thing is worth saying about a suite that passed on its first run. The C++ implementation's first run failed eight integration assertions, and that failure found a real contradiction between two specification sections. A first run that passes says the code matches a specification written hours earlier by the same author, which is a narrower claim than a first run that fails and is then reconciled. The Process page makes the same point about this thread's evidence as a whole.

7.  Prompt Records

This page carries none. Page_Structure.md §8 assigns it a Prompts_CSharp_Spec_driven_TextFinder_Tests.md, and this thread produced one record covering every turn, which sits on the Process page. The turn that wrote these four suites and the three runners is summarized there.