3. PageValidator Project
PageValidator checks HTML files for structural correctness. Four implementations -
Rust, C++, C#, and Python - share the same four-component pipeline architecture,
identical validation rules, and a uniform command-line interface, making the project a
controlled cross-language comparison.
Why PageValidator is useful for language comparison
-
The workload is CPU-bound character parsing, not I/O-bound regex scanning.
That reverses the language-ranking from TextFinder and makes the performance
story interesting in a different way.
-
All four variants produce identical pass/fail reports, so correctness is easy
to verify by diffing output against a reference run.
-
The four-stage pipeline (Tokenizer → Lexer → Validator → EntryPoint)
maps cleanly onto different language idioms -
enum /
std::variant / abstract record / base-class hierarchy
- for the same logical type.
-
The spec-driven workflow (Constitution.md, Structure.md, Spec.md) was used
to build each variant, so you can compare how the same spec translates
into different language idioms.
3.1 Architecture
Each implementation is a four-component pipeline with a strictly linear dependency chain.
No component depends on anything to its right; the three library components use only
the language's standard library.
Tokenizer <-- Lexer <-- Validator <-- EntryPoint
- Tokenizer - reads raw HTML source and emits a flat stream of coarse tokens. Knows only
<, >, =, ", ', !, /, -. No HTML grammar.
- Lexer - consumes the token stream and groups tokens into structured lexemes (open tags, close tags, attributes, text, comments, doctype) with source positions. Lowercases all tag names.
- Validator - drives the lexer, maintains an open-tag stack, applies the eight structural rules, and returns a report containing every error found. Never short-circuits.
- EntryPoint - parses command-line flags, iterates HTML files, calls the Validator, and prints a human-readable pass/fail report.
The Rust variant expresses each pipeline stage as a public struct behind a thin
lib.rs facade. The Validator's core loop is a single match over
incoming Lexemes:
pub struct ValidationError {
pub rule: &'static str,
pub message: String,
pub line: usize,
pub col: usize,
}
pub struct Report {
pub file: PathBuf,
pub errors: Vec<ValidationError>,
}
impl Report {
pub fn is_valid(&self) -> bool { self.errors.is_empty() }
}
3.2 Command-Line Interface
All four implementations share the same flags and output format.
The tool accepts one or more paths; directories are walked recursively
only when -r is supplied.
| Option |
Meaning |
Default |
<path>... |
One or more HTML files or directories to validate |
(required) |
-r, --recursive |
Descend into subdirectories |
off |
-q, --quiet |
Print only files with errors; suppress PASS lines |
off |
-s, --summary |
Append a pass/fail count after all files are processed |
off |
-h, --help |
Print help and exit |
off |
Example - validate an entire site tree, suppress PASS lines, print a count:
rs_page_validator -r -q -s ./site
Output format - each file is reported as PASS or FAIL;
failing files list every violation with rule name, line, and column:
PASS site/index.html
FAIL site/about.html
[tag-nesting] 14:3 -- </div> does not match open <p>
[attr-quotes] 22:9 -- attribute 'class' value 'hero' is not quoted
PASS site/contact.html
2 passed, 1 failed
3.3 Skip Lists
EntryPoint in every variant skips a hard-coded set of directory names during recursive
traversal. Entries are matched against the bare directory name, so a target
folder at any depth is skipped regardless of where it appears.
| Category |
Skipped directory names |
| Build artifacts |
bin, obj (C#); target (Rust); build, out (C++) |
| Python |
__pycache__, .venv, venv, dist |
| VCS / IDE |
.git, .vs, .idea |
| Archives |
archive |
3.4 Validation Rules
The Validator applies eight rules in a single pass over the lexeme stream.
All violations are collected before the report is returned - the
Validator never aborts early on the first error.
| Rule |
What is checked |
doctype |
Document begins with <!DOCTYPE html> |
root-element |
Exactly one <html> element wraps the entire document |
head-required |
<head> is present and contains at least one <title> |
body-required |
<body> element is present |
tag-nesting |
Every open tag has a matching close tag in the correct stack order |
void-elements |
Void elements (br, hr, img, input, link, meta, …) carry no close tag |
attr-quotes |
All attribute values are enclosed in single or double quotes |
duplicate-id |
The id attribute value is unique within the document |
3.5 Language Implementations
| Project |
Language |
Build tool |
Entry point |
| CppPageValidator |
C++23 (named modules) |
CMake 3.28+ / MSVC or Clang |
build\entry_point\Release\page_validator |
| CsPageValidator |
C# 12 / .NET 10 |
dotnet SDK |
dotnet run --project EntryPoint -- |
| PyPageValidator |
Python 3.10+ |
none (run directly) |
python EntryPoint/page_validator.py |
| rs_page_validator |
Rust 2021 (Cargo workspace) |
cargo |
cargo run -- from rs_page_validator/ |
Each project also carries a generate_part.py script that calls the
Claude API to regenerate any component from its Spec.md:
python generate_part.py Tokenizer
python generate_part.py Lexer
python generate_part.py Validator
python generate_part.py EntryPoint
3.6 Performance
Timings from pa_timer.py against the full NewSite HTML tree.
20 timed runs per validator; first run discarded as warm-up.
Median = average of the 10th and 11th sorted values.
664 files visited; 425 fail at least one rule.
| Validator |
Min (s) |
Median (s) |
Max (s) |
| CppPageValidator |
0.521 |
0.645 |
2.729 |
| rs_page_validator |
0.901 |
0.936 |
1.972 |
| CsPageValidator |
1.127 |
1.290 |
1.538 |
| PyPageValidator |
2.766 |
2.846 |
3.029 |
All four agree on 664 files and 425 failures, confirming behavioral equivalence.
The C++ max outlier (2.729 s) reflects an OS scheduling interrupt during that
run; the median is representative of steady-state cost.
Why the ranking reverses from TextFinder
-
The workload is CPU-bound, not I/O-bound.
TextFinder reads large source files and runs a regex engine; most time is
spent in OS calls. PageValidator reads small HTML files and runs a
character-by-character tokenizer; most time is spent executing the
parser logic itself. That shifts the bottleneck from the kernel to
the language runtime.
-
C++ wins because there is no regex tax.
The tokenizer scans one character at a time with no regular expressions,
so the NFA penalty that hurt C++ in TextFinder does not apply here.
MSVC's optimizer inlines the four pipeline stages aggressively, and
std::variant dispatch is resolved at compile time.
-
Rust is close but slightly behind C++.
The four-crate compilation boundary limits cross-crate inlining compared
to C++'s single-TU build. The Rust validator is otherwise equivalent
in algorithm to the C++ version.
-
C# pays .NET startup cost.
The runtime and JIT spin up before the first file is touched, consuming
a large fraction of elapsed time for this short-running tool.
Steady-state throughput after warmup is comparable to Rust.
-
Python is slowest because the hot path is pure Python.
TextFinder's hot path was C extension calls (
os.scandir,
re.search). Here the hot path is the character-by-character
tokenizer loop, which executes entirely in the Python interpreter.
3.7 Language Comparison
The same logical types appear in all four variants. The table shows how each
language expresses the key abstractions.
| Aspect |
Rust |
C++ |
C# |
Python |
| Token type |
enum Token |
std::variant |
abstract record |
base class + @dataclass |
| Lexeme type |
enum Lexeme |
std::variant |
abstract record |
base class + @dataclass |
| Null / absence |
Option<Lexeme> |
std::optional<Lexeme> |
Lexeme? |
Lexeme | None |
| Test runner |
cargo test |
custom test.cpp |
custom Exe runner |
unittest |
| External deps |
none (std only) |
none (std only) |
none (BCL only) |
none (stdlib only) |
3.8 Code Metrics
Generated by code_metrics.py from the Projects directory.
Lines = total line count (code + comments + blanks).
Scopes = scope-opening tokens: { count for brace
languages; lines ending with : for Python.
rs_page_validator (Rust)
| File |
Lines |
Scopes |
| entry_point/src/main.rs | 135 | 42 |
| lexer/src/lib.rs | 185 | 62 |
| tokenizer/src/lib.rs | 312 | 65 |
| validator/src/lib.rs | 270 | 64 |
| TOTAL | 902 | 233 |
CppPageValidator (C++)
| File |
Lines |
Scopes |
| entry_point/src/main.cpp | 161 | 20 |
| lexer/src/Lexer.ixx | 144 | 41 |
| lexer/src/test.cpp | 138 | 29 |
| tokenizer/src/test.cpp | 157 | 28 |
| tokenizer/src/Tokenizer.ixx | 185 | 58 |
| validator/src/test.cpp | 137 | 29 |
| validator/src/Validator.ixx | 176 | 30 |
| TOTAL | 1098 | 235 |
CsPageValidator (C#)
| File |
Lines |
Scopes |
| EntryPoint/Program.cs | 170 | 31 |
| Lexer/Lexer.cs | 129 | 12 |
| Lexer.Tests/Tests.cs | 115 | 29 |
| Tokenizer/Tokenizer.cs | 195 | 25 |
| Tokenizer.Tests/Tests.cs | 133 | 32 |
| Validator/Validator.cs | 151 | 23 |
| Validator.Tests/Tests.cs | 129 | 18 |
| TOTAL | 1022 | 170 |
PyPageValidator (Python)
| File |
Lines |
Scopes |
| EntryPoint/page_validator.py | 155 | 39 |
| Lexer/lexer.py | 126 | 32 |
| Lexer/test_lexer.py | 83 | 13 |
| Tokenizer/tokenizer.py | 197 | 52 |
| Tokenizer/test_tokenizer.py | 108 | 15 |
| Validator/validator.py | 159 | 34 |
| Validator/test_validator.py | 91 | 15 |
| TOTAL | 919 | 200 |
3.9 References