11.  PageValidator Project

PageValidator checks HTML files for structural correctness. Four implementations - Rust, C++, C#, and Python - share the same four-component pipeline architecture, identical validation rules, and a uniform command-line interface, making the project a controlled cross-language comparison.

Why PageValidator is useful for language comparison

1. The workload is CPU-bound character parsing, not I/O-bound regex scanning. That reverses the language-ranking from TextFinder and makes the performance story interesting in a different way.
2. All four variants produce identical pass/fail reports, so correctness is easy to verify by diffing output against a reference run.
3. The four-stage pipeline (Tokenizer → Lexer → Validator → EntryPoint) maps cleanly onto different language idioms - enum / std::variant / abstract record / base-class hierarchy - for the same logical type.
4. The spec-driven workflow (Constitution.md, Structure.md, Spec.md) was used to build each variant, so you can compare how the same spec translates into different language idioms.

11.1  Architecture

Each implementation is a four-component pipeline with a strictly linear dependency chain. No component depends on anything to its right; the three library components use only the language's standard library.

Tokenizer <-- Lexer <-- Validator <-- EntryPoint

• Tokenizer - reads raw HTML source and emits a flat stream of coarse tokens. Knows only <, >, =, ", ', !, /, -. No HTML grammar.
• Lexer - consumes the token stream and groups tokens into structured lexemes (open tags, close tags, attributes, text, comments, doctype) with source positions. Lowercases all tag names.
• Validator - drives the lexer, maintains an open-tag stack, applies the eight structural rules, and returns a report containing every error found. Never short-circuits.
• EntryPoint - parses command-line flags, iterates HTML files, calls the Validator, and prints a human-readable pass/fail report.

The Rust variant expresses each pipeline stage as a public struct behind a thin lib.rs facade. The Validator's core loop is a single match over incoming Lexemes:

pub struct ValidationError {
    pub rule: &'static str,
    pub message: String,
    pub line: usize,
    pub col: usize,
}

pub struct Report {
    pub file: PathBuf,
    pub errors: Vec<ValidationError>,
}

impl Report {
    pub fn is_valid(&self) -> bool { self.errors.is_empty() }
}

11.2  Command-Line Interface

All four implementations share the same flags and output format. The tool accepts one or more paths; directories are walked recursively only when -r is supplied.

Option	Meaning	Default
<path>...	One or more HTML files or directories to validate	(required)
-r, --recursive	Descend into subdirectories	off
-q, --quiet	Print only files with errors; suppress PASS lines	off
-s, --summary	Append a pass/fail count after all files are processed	off
-h, --help	Print help and exit	off

Example - validate an entire site tree, suppress PASS lines, print a count:

rs_page_validator -r -q -s ./site

Output format - each file is reported as PASS or FAIL; failing files list every violation with rule name, line, and column:

PASS  site/index.html
FAIL  site/about.html
      [tag-nesting] 14:3 -- </div> does not match open <p>
      [attr-quotes] 22:9 -- attribute 'class' value 'hero' is not quoted
PASS  site/contact.html

2 passed, 1 failed

11.3  Skip Lists

EntryPoint in every variant skips a hard-coded set of directory names during recursive traversal. Entries are matched against the bare directory name, so a target folder at any depth is skipped regardless of where it appears.

Category	Skipped directory names
Build artifacts	bin, obj (C#); target (Rust); build, out (C++)
Python	__pycache__, .venv, venv, dist
VCS / IDE	.git, .vs, .idea
Archives	archive

11.4  Validation Rules

The Validator applies eight rules in a single pass over the lexeme stream. All violations are collected before the report is returned - the Validator never aborts early on the first error.

Rule	What is checked
doctype	Document begins with <!DOCTYPE html>
root-element	Exactly one <html> element wraps the entire document
head-required	<head> is present and contains at least one <title>
body-required	<body> element is present
tag-nesting	Every open tag has a matching close tag in the correct stack order
void-elements	Void elements (br, hr, img, input, link, meta, …) carry no close tag
attr-quotes	All attribute values are enclosed in single or double quotes
duplicate-id	The id attribute value is unique within the document

11.5  Language Implementations

Project	Language	Build tool	Entry point
CppPageValidator	C++23 (named modules)	CMake 3.28+ / MSVC or Clang	build\entry_point\Release\page_validator
CsPageValidator	C# 12 / .NET 10	dotnet SDK	dotnet run --project EntryPoint --
PyPageValidator	Python 3.10+	none (run directly)	python EntryPoint/page_validator.py
rs_page_validator	Rust 2021 (Cargo workspace)	cargo	cargo run -- from rs_page_validator/

Each project also carries a generate_part.py script that calls the Claude API to regenerate any component from its Spec.md:

python generate_part.py Tokenizer
python generate_part.py Lexer
python generate_part.py Validator
python generate_part.py EntryPoint

11.6  Performance

Timings from pa_timer.py against the full NewSite HTML tree. 20 timed runs per validator; first run discarded as warm-up. Median = average of the 10th and 11th sorted values. 664 files visited; 425 fail at least one rule.

Validator	Min (s)	Median (s)	Max (s)
CppPageValidator	0.521	0.645	2.729
rs_page_validator	0.901	0.936	1.972
CsPageValidator	1.127	1.290	1.538
PyPageValidator	2.766	2.846	3.029

All four agree on 664 files and 425 failures, confirming behavioral equivalence. The C++ max outlier (2.729 s) reflects an OS scheduling interrupt during that run; the median is representative of steady-state cost.

Why the ranking reverses from TextFinder

• The workload is CPU-bound, not I/O-bound. TextFinder reads large source files and runs a regex engine; most time is spent in OS calls. PageValidator reads small HTML files and runs a character-by-character tokenizer; most time is spent executing the parser logic itself. That shifts the bottleneck from the kernel to the language runtime.
• C++ wins because there is no regex tax. The tokenizer scans one character at a time with no regular expressions, so the NFA penalty that hurt C++ in TextFinder does not apply here. MSVC's optimizer inlines the four pipeline stages aggressively, and std::variant dispatch is resolved at compile time.
• Rust is close but slightly behind C++. The four-crate compilation boundary limits cross-crate inlining compared to C++'s single-TU build. The Rust validator is otherwise equivalent in algorithm to the C++ version.
• C# pays .NET startup cost. The runtime and JIT spin up before the first file is touched, consuming a large fraction of elapsed time for this short-running tool. Steady-state throughput after warmup is comparable to Rust.
• Python is slowest because the hot path is pure Python. TextFinder's hot path was C extension calls (os.scandir, re.search). Here the hot path is the character-by-character tokenizer loop, which executes entirely in the Python interpreter.

11.7  Language Comparison

The same logical types appear in all four variants. The table shows how each language expresses the key abstractions.

Aspect	Rust	C++	C#	Python
Token type	enum Token	std::variant	abstract record	base class + @dataclass
Lexeme type	enum Lexeme	std::variant	abstract record	base class + @dataclass
Null / absence	Option<Lexeme>	std::optional<Lexeme>	Lexeme?	Lexeme | None
Test runner	cargo test	custom test.cpp	custom Exe runner	unittest
External deps	none (std only)	none (std only)	none (BCL only)	none (stdlib only)

11.8  Code Metrics

Generated by code_metrics.py from the Projects directory. Lines = total line count (code + comments + blanks). Scopes = scope-opening tokens: { count for brace languages; lines ending with : for Python.

rs_page_validator (Rust)

File	Lines	Scopes
entry_point/src/main.rs	135	42
lexer/src/lib.rs	185	62
tokenizer/src/lib.rs	312	65
validator/src/lib.rs	270	64
TOTAL	902	233

CppPageValidator (C++)

File	Lines	Scopes
entry_point/src/main.cpp	161	20
lexer/src/Lexer.ixx	144	41
lexer/src/test.cpp	138	29
tokenizer/src/test.cpp	157	28
tokenizer/src/Tokenizer.ixx	185	58
validator/src/test.cpp	137	29
validator/src/Validator.ixx	176	30
TOTAL	1098	235

CsPageValidator (C#)

File	Lines	Scopes
EntryPoint/Program.cs	170	31
Lexer/Lexer.cs	129	12
Lexer.Tests/Tests.cs	115	29
Tokenizer/Tokenizer.cs	195	25
Tokenizer.Tests/Tests.cs	133	32
Validator/Validator.cs	151	23
Validator.Tests/Tests.cs	129	18
TOTAL	1022	170

PyPageValidator (Python)

File	Lines	Scopes
EntryPoint/page_validator.py	155	39
Lexer/lexer.py	126	32
Lexer/test_lexer.py	83	13
Tokenizer/tokenizer.py	197	52
Tokenizer/test_tokenizer.py	108	15
Validator/validator.py	159	34
Validator/test_validator.py	91	15
TOTAL	919	200

11.9  References

Resource	Description
Code Story: Spec-Driven Development	The workflow used to generate each component from its Spec.md.
Code Story: Experiment Arena	Timing tools and how to interpret the performance numbers.
Code Story: TextFinder	A companion project; compare the performance ranking reversal between the two projects.