3. RustDirNav — DirNav<App: DirEvent>
RustDirNav is a generic depth-first directory walker. It is
parameterized by an application type App that must implement
the DirEvent trait — two callbacks that fire when the walker
enters a directory and when it encounters a matching file:
pub trait DirEvent {
fn do_dir(&mut self, d: &str);
fn do_file(&mut self, f: &str);
}
This trait is the entire contract between navigator and application.
Every rs_textfinder_opt-specific behavior — regex matching,
hide/show, path-relative printing — lives in TfAppl, an
implementor supplied by the binary crate. The library itself remains
free of any TextFinder concepts and could equally well be reused by a
line-counting tool, a build-artifact scrubber, or a checksum walker.
3.1 Design Points
-
Skip list is baked in. Build directories
(
target, bin, obj,
build, out), Python caches, VCS/IDE folders,
and archive are pre-loaded into skip_dirs
at construction so no callers have to know the list. Extra names
can be added via add_skip().
-
DirEntry::file_type(), not
Path::is_dir(). The type is already carried by
the readdir response on both Windows and Unix, so no
extra stat syscall is issued per entry. This single
call was the dominant optimization gain over the baseline
rs_textfinder — median elapsed time dropped from
~0.91 s to ~0.61 s, a 33% improvement.
-
Directory printed only when it has matches. Files
are accumulated in a
Vec during the scan pass; the
directory callback fires only if that vector is non-empty (or if
hide is off). This inverts the natural traversal order
but produces cleaner grouped output.
-
Windows separator normalization. Paths are
converted to forward slashes at the emit site, so downstream
printing and regex matching see one path style regardless of host OS.
-
App owned by DirNav.
DirNav<App>
constructs an instance of App via App::default()
and owns it for the lifetime of the walker. get_app()
returns a mutable reference so callers can configure the app (regex,
hide flag) before visit().
3.2 visit() — the traversal loop
visit() is a straight recursion. For each entry produced by
fs::read_dir:
- Grab the cached
file_type() — no extra syscall.
- If it is a directory, check the skip list; if not skipped, buffer
its normalized path for the recursion pass.
- If it is a file and its extension is in the pattern list (or the
pattern list is empty), buffer its name and bump the file counter.
- After the scan, fire
do_dir only if the buffered file
list is non-empty or hide is off; fire
do_file for each buffered file.
- Recurse into each buffered subdirectory if
recurse is
on.
Separating the file-emit pass from the directory-emit decision is what
lets hide=true suppress the directory header for
no-match folders without a look-ahead: by the time
do_dir is called, the walker already knows whether any file
will follow.
3.3 Source — RustDirNav/src/dir_nav_lib.rs
/////////////////////////////////////////////////////////////
// dir_nav_lib.rs //
// //
// Jim Fawcett, https://JimFawcett.github.io, 12 Apr 2020 //
/////////////////////////////////////////////////////////////
/*
DirNav<App> is a directory navigator that uses the generic
parameter App to define how files and directories are
handled.
- displays only paths that have file targets by default
- hide(false) will show all directories traversed
- recurses directory tree at specified root by default
- recurse(false) examines only specified path.
*/
use std::fs::{self, DirEntry};
use std::io;
use std::io::{Error, ErrorKind};
#[allow(unused_imports)]
use std::path::{Path, PathBuf};
/// trait required of the App generic parameter type
pub trait DirEvent {
fn do_dir(&mut self, d: &str);
fn do_file(&mut self, f: &str);
}
/////////////////////////////////////////////////
// Patterns are a collection of extension strings
// used to identify files as search targets
type SearchPatterns = Vec<std::ffi::OsString>;
/// Directory Navigator Structure
#[allow(dead_code)]
#[derive(Debug, Default)]
pub struct DirNav<App: DirEvent> {
/// file extensions to look for
pats: SearchPatterns,
/// directory names to skip during traversal
skip_dirs: SearchPatterns,
/// instance of App : DirEvent, requires do_file and do_dir methods
app: App,
/// number of files processed
num_file: usize,
/// number of dirs processed
num_dir: usize,
/// recurse ?
recurse : bool,
/// hide dirs with no targets ?
hide: bool,
}
impl<App: DirEvent + Default> DirNav<App> {
pub fn new() -> Self
where
App: DirEvent + Default,
{
let defaults = [
// C#/.NET
"bin", "obj",
// Rust
"target",
// C++
"build", "out",
// Python
"__pycache__", ".venv", "venv", "dist",
// VCS / IDE
".git", ".vs", ".idea",
// archives
"archive",
];
let mut skip_dirs = SearchPatterns::new();
for name in &defaults {
let mut s = std::ffi::OsString::new();
s.push(name);
skip_dirs.push(s);
}
Self {
pats: SearchPatterns::new(),
skip_dirs,
app: App::default(),
num_file: 0,
num_dir: 0,
recurse: true,
hide: true,
}
}
/// do recursive visit?
pub fn recurse(&mut self, p:bool) {
self.recurse = p;
}
/// hide dirs with no targets?
pub fn hide(&mut self, p:bool) {
self.hide = p;
}
/// return reference to App to get results, if any
pub fn get_app(&mut self) -> &mut App {
&mut self.app
}
/// return number of dirs processed
pub fn get_dirs(&self) -> usize {
self.num_dir
}
/// return number of files processed
pub fn get_files(&self) -> usize {
self.num_file
}
/// return patterns, e.g., file extensions to look for
pub fn get_patts(&self) -> &SearchPatterns {
&self.pats
}
/// add directory name to skip during traversal - takes either String or &str
pub fn add_skip<S: Into<String>>(&mut self, s: S) -> &mut DirNav<App> {
let mut t = std::ffi::OsString::new();
t.push(s.into());
self.skip_dirs.push(t);
self
}
/// add extension to search for - takes either String or &str
pub fn add_pat<S: Into<String>>(&mut self, p: S) -> &mut DirNav<App> {
let mut t = std::ffi::OsString::new();
t.push(p.into());
self.pats.push(t);
self
}
/// reset to default state
pub fn clear(&mut self) {
self.pats.clear();
self.num_dir = 0;
self.num_file = 0;
self.app = App::default();
}
/// Depth First Search for file extensions starting at path dir<br />
/// Displays only directories with files matching pattern
pub fn visit(&mut self, dir: &Path) -> io::Result<()>
where App: DirEvent
{
self.num_dir += 1;
let dir_name: String =
self.replace_sep(dir).to_string_lossy().to_string();
let mut files = Vec::<std::ffi::OsString>::new();
let mut sub_dirs = Vec::<std::ffi::OsString>::new();
if dir.is_dir() {
/* search local directory */
for entry in fs::read_dir(dir)? {
let entry = entry?;
let file_type = entry.file_type()?;
if file_type.is_dir() {
let name = entry.file_name();
let skip = self.skip_dirs.contains(&name);
if !skip {
let cd = self.replace_sep(&entry.path());
sub_dirs.push(cd);
}
} else if file_type.is_file() {
if self.in_patterns(&entry) | self.pats.is_empty() {
self.num_file += 1;
files.push(entry.file_name());
}
}
}
/*-- display only dirs with found files --*/
if !files.is_empty() || !self.hide {
self.app.do_dir(&dir_name);
}
for fl in files {
let flnm = fl.to_string_lossy().to_string();
self.app.do_file(&flnm);
}
/*-- recurse into subdirectories --*/
for sub in sub_dirs {
let mut pb = std::path::PathBuf::new();
pb.push(sub);
if self.recurse {
self.visit(&pb)?;
}
}
return Ok(()); // normal return
}
Err(Error::new(ErrorKind::Other, "not a directory"))
}
/// replace Windows directory separator with Linux separator
pub fn replace_sep(&self, path: &Path) -> std::ffi::OsString {
let rtn = path.to_string_lossy();
let mod_path = rtn.replace("\\", "/");
let mut os_str: std::ffi::OsString = std::ffi::OsString::new();
os_str.push(mod_path);
os_str
}
/// does store contain d.path().extension() ?
pub fn in_patterns(&self, d: &DirEntry) -> bool {
let p = d.path();
let ext = p.extension();
match ext {
Some(extn) => self.pats.contains(&(extn.to_os_string())),
None => false,
}
}
}
#[cfg(test)]
mod tests {
// test_setup() should run first. To ensure that:
// use cargo -- --test-threads=1
// to see console output:
// use cargo test -- --show-output --test-threads=1
use super::*;
#[derive(Debug)]
struct ApplTest {
rslt_store: Vec<String>,
}
impl DirEvent for ApplTest {
fn do_dir(&mut self, _d: &str) {
}
fn do_file(&mut self, f: &str) {
self.rslt_store.push((*f).to_string());
}
}
impl Default for ApplTest {
fn default() -> Self {
ApplTest {
rslt_store: Vec::<String>::new(),
}
}
}
#[test]
fn test_setup() {
let _ = std::fs::create_dir("./test_dir");
let _ = std::fs::create_dir("./test_dir/test_sub1_dir");
let _ = std::fs::create_dir("./test_dir/test_sub2_dir");
let _ = std::fs::File::create("./test_dir/test_file.rs");
let _ = std::fs::File::create("./test_dir/test_sub1_dir/test_file1.rs");
let _ = std::fs::File::create("./test_dir/test_sub1_dir/test_file2.exe");
let _ = std::fs::File::create("./test_dir/test_sub2_dir/test_file3.txt");
}
#[test]
fn test_walk() {
let mut dn = DirNav::<ApplTest>::new();
dn.add_pat("rs").add_pat("exe").add_pat("txt");
let mut pb = PathBuf::new();
pb.push("./test_dir".to_string());
let _ = dn.visit(&pb);
let rl = &dn.get_app().rslt_store;
print!("\n {:?}", rl);
let l = |s: &str| -> String { s.to_string() };
assert!(rl.contains(&l("test_file.rs")));
assert!(rl.contains(&l("test_file1.rs")));
assert!(rl.contains(&l("test_file2.exe")));
assert!(rl.contains(&l("test_file3.txt")));
}
#[test]
fn test_patts() {
let mut dn = DirNav::<ApplTest>::new();
dn.add_pat("foo").add_pat("bar");
assert_eq!(dn.get_patts().len(), 2);
let pats = dn.get_patts();
let mut foo_str = std::ffi::OsString::new();
foo_str.push("foo");
assert!(pats.contains(&foo_str));
let mut bar_str = std::ffi::OsString::new();
bar_str.push("bar");
assert!(pats.contains(&bar_str));
dn.clear();
assert_eq!(dn.get_patts().len(), 0);
}
}