Rust Bites: Macros

declarative macros support variadics

About Rust Track

1.0  Introduction

Macros allow users to extend Rust syntax by replacing a macro declaration with code specified by the macro body. There are two useful consequences of using macros:
  1. Code can be writen once and applied in many places. Functions do that too, but macros create code in the same scope as the macro declaration, where functions create code in a child scope. That means that a locally defined macro can use local data - very like cut and paste code, but with the advantage that there is only one source for change.
  2. Macros can be variadic, i.e., accept an arbitrary number of arguments. That is a consequence of pattern matching used to define macros.
Locally defined macro fn main() { let offset = 5; macro_rules! add_with_local_offset { ($left:expr , $right:expr) => { $left + $right + offset }; } let (a, b) = (1, 2); let result = add_with_local_offset!(a, b); let s = stringify!(1 + 2 + offset); print!("\n {:?} = {:?}", s, result); } Output - locally defined macro "1 + 2 + offset" = 8 Variadic macro /* Demonstrate variadic macros */ macro_rules! variadic_macro { /* terminating rule */ ($h:expr) => { print!("\n processing argument {}", $h); }; /* process arguments recursively */ ($h:expr, $($t:expr),+) => { /* process head */ variadic_macro! { $h } /* process tail */ variadic_macro! { $($t),+ } }; } fn main() { variadic_macro!(1, 2, 3, 4); println!(); variadic_macro!('a', 'b', 'c'); } Output - Variadic macro processing argument 1 processing argument 2 processing argument 3 processing argument 4 processing argument a processing argument b processing argument c
Local Macro in playground
Variadic Macro in playground

1.0  Basics:

Rust macros come in two flavors: declarative and procedural. Procedural macros support defining custom #[derive] macros, function-like macros that run at compile-time, and custom attributes. They can be defined in a crate, loaded by Rust and executed as the macro is expanded. Procedure macros operate on proc_macro::TokenStreams. Those are iterators over TokenTrees. A TokenTree contains identifiers, literals, punctuators, and groups. Developing a proc macro involves parsing TokenTrees. We won't consider procedural macros further in this Bite. Instead we focus on declarative macros. They take the form:
Macro structure macro_rules! name { rule0 ; rule1 ; // ... ruleN ; } Rule structure (pattern) => {expansion}
They are composed of one or more rules that describe how to generate code based on their inputs. Each rule has a pattern matcher and an associated block that generates code from the pattern. Here's an example:
Indented print macro macro_rules! indent_print { /* first rule */ /* pattern */ ($a:expr) => { { /* expansion template */ print!("\n {}", $a) } }; /* second rule */ () => { { print!("\n no argument to print") } } } Using code fn main() { indent_print!("this is a demo"); indent_print!(); } Output this is a demo no argument to print
It isn't very useful except to illustrate how pattern matching works. In the first rule the pattern is an expression, &a, that is expanded using a print! macro, just displaying the input. The second rule has no input and simply prints out that fact.

2.0  Variadic Macros

Patterns and expansions support repetitions using a syntax reminiscent of regular expressions. We saw this in the second opening example.

Table 1. - Repetition Syntax

Syntax Semantics
$(...)* repeat zero or more ... with no seperator
$(...),* repeat zero or more ... with comma seperator
$(...)+ repeat one or more ... with no seperator
$(...),+ repeat one or more ... with comma seperator
The Example below extends our indent_print macro to accept an arbitry number of input arguments and simply passes the input repeated sequence to a print macro.
Indented print with multiple arguments macro_rules! ma_indent_print { /*-------------------------------------------- pattern matches $fmt expression followed by zero or more expressions --------------------------------------------*/ ($fmt:expr, $($a:expr),*) => { /*---------------------------------------- template creates $fmt followed by zero or more expressions follwed by commas ----------------------------------------*/ print!($fmt, $($a),*) // each $a taken by ref } } Using code fn main() { let s = String::from("Rust"); ma_indent_print!( "\n {}, {}, {}", "hi", 42, s ); /* s taken by ref so not moved */ print!("\n {}", s); } Output hi, 42, Rust Rust
That is not too complicated. But we need to explore how repetitive expansions are handled, rather than simply passing them along to another variadic macro like print. That we do in the next example.
code in playground

2.1  Practical Example:

The next example uses inputs more structured than simple expressions. Table 2 provides a partial list of types that declarative macros can accept and process.

Table 2. - Partial list of Macro Fragments

Fragment Type Matches Examples
expr an expression 1 + 2, v.len(), "a string"
ident an identifier x, my_struct
ty a type String, Vec<String>
path a std::path C:\\temp\\CodeExperiments
pat a pattern _, Some(ref x)
literal a literal value 2, "a literal string"
tt token tree let x=2; let y=x;, [0, 1, 2], fn f{...}
In this example we will construct a timer that accepts a rich set of inputs for which we want performance data. The inputs are a literal that displays what we are timing and a token tree, tt, that allows us to specify statements, functions, and arbitrary combinations of them.
Timer Macro use std::{thread, time}; use std::result::*; use std::io::prelude::*; use std::fs::File; macro_rules! time_it { /* measure time to process token tree &tt */ ($context:literal, $($tt:tt)*) => { let timer = std::time::Instant::now(); $( $tt )* let t = timer.elapsed(); println!("{}: {:?}", $context, t); }; /* measure time to scaffold without execution */ ($context:literal) => { let timer = std::time::Instant::now(); let t = timer.elapsed(); println!("{}: {:?}", $context, t); }; } Output empty: 928ns let _x = 1 + 2: 956ns mults: 2.636µs sleep(10) + sleep(20): 30.241214ms file open: 139.476µs Using Code fn do_sleep(ms:u64) { let millis = time::Duration::from_millis(ms); thread::sleep(millis); } fn open_file_for_write(file_name:&str) ->Result<(), std::io::Error> { let mut file = File::create(file_name)?; file.write_all(b"Hello, Rust BuildOn!")?; Ok(()) } fn main() { time_it!("empty"); time_it!( "let _x = 1 + 2", let _x = 1 + 2; ); time_it!( "mults", let mut x:f64 = 1.5; for _i in 0..10 { x *= x; } ); time_it!( "sleep(10) + sleep(20)", do_sleep(10); do_sleep(20); ); time_it!( "file open", let _ = open_file_for_write("foobar"); ); }
Token trees, tt, may include tokens that are not expressions, like identifiers. We could add another rule that ignores them, like the empty rule. If a user supplies an identifier as an input, that causes a compiler warning to be emitted, but no build or execution errors. Since it is clear that an identifier has no inherent execution I chose not to provide that rule.
code in playground
Here is another useful macro example that presents a variadic macro:
Macro for Creating Hashmaps use std::collections::HashMap; macro_rules! hash_map { /* expr can be separated only by , ; or => */ ($( $key:expr , $value:expr ),*) => { { let mut h = HashMap::new(); $( h.insert($key, $value); )* h } }; } fn main() { let map = hash_map!( "zero" , 0, "one" , 1, "two" , 2, "three" , 3, "four" , 4 ); print!("\n {:?}", map); } Output { "two": 2, "four": 4, "zero": 0, "one": 1, "three": 3 }
code in playground

3.0 Standard Macros

Rust provides a number of frequently used macros. Many of these we have encountered in previous bites:

Table 3. - Partial list of Standard Macros

std Macro Description Example
assert Asserts that boolean expression is true assert!(x==y, "x != y");
assert_eq Asserts that expressions are equal assert_eq!(x, y, msg);
assert_ne Asserts expressions are not equal assert_ne!(x, y, msg);
concat concatenates literals into static string slice let x = concat!(a,2.5,"lit_str");
debug prints and returns value of expression dbg!(a*2);
file returns name of containing file file!();
format same as print! except returns String let s = format!("{:?}",x);
line returns number of containing line let n = line!();
panic panics current thread panic!();
print Prints to std out print!(frm, x, y, ...);
stringify converts argument symbols to strings and concatenates stringify!(a + b, "lit str");
vec creates new vector let v = vec![0, 1, 2];
write writes formatted data to buffer write!(&mut buff, fmt_str, x, y, ...);

4.0  Final Thoughts:

One issue with macros that is easy to overlook: macro processing may move non-copy types so they become invalid after executing the offending macro. In the example below, we provide two versions of a demonstration macro. The first, show_compare1, uses a rust match operation that takes its arguments by value, consuming them. So use of its arguments after macro execution will cause a build failure. The second version, show_compare2 passes its arguments by reference for matching and does not invalidate its arguments. You can open the playground example and experiment with calling either one for both copy and non-copy arguments.
Macros #![allow(unused_macros)] /*---------------------------------------- macro show_compare1 will move its args if they are not copy ----------------------------------------*/ macro_rules! show_compare1 { ($left:expr , $right:expr) => { match ($left, $right) { (left_val, right_val) => { if(left_val == right_val) { print!( "\n {:?} == {:?}", left_val, right_val ); } else { print!( "\n {:?} != {:?}", left_val, right_val ); } } } }; } /*---------------------------------------- macro show_compare2 will not move its args even if they are not copy ----------------------------------------*/ macro_rules! show_compare2 { ($left:expr , $right:expr) => { match (&$left, &$right) { (left_val, right_val) => { if(*left_val == *right_val) { print!( "\n {:?} == {:?}", left_val, right_val ); } else { print!( "\n {:?} != {:?}", left_val, right_val ); } } } }; } Using Code fn main() { // let (a, b) = (1, 2); let a = String::from("a string"); let mut b = a.clone(); b.push_str(" and another"); // let a = vec![1, 2, 3]; // let mut b = a.clone(); // b.push(4); show_compare2!(a,b); /*--------------------------------------- Statements below fail to compile if a or b are moved. ---------------------------------------*/ print!("\n {:?}", a); print!("\n {:?}", b); } Output "a string" != "a string and another" "a string" "a string and another"
code in playground

5.0  References:

Ref Link Description
Macros By Example Part of the Rust reference, describes declarative syntax with precise BNF productions.
Importing macros Directions for importing macros in Rust edition-guide for rust-2018
time_it: a Case Study in Rust Macros. The timer example above has modifications of code presented here.
rust by example Collection of annotated macro examples
The Little Book of Rust Macros Presents background details and some usefule macro patterns.
Macros in Rust: A tutorial with examples Presents examples of both Declarative and Procedural macros.
Procedural Macros in Rust 2018 Presents ideas, but no fully formed examples.