Storage

For now only a key-value store is supported for persistent storage. Eventually, additional facilities may be added: a cyclic logging journal, a file-system, raw flash access, etc.

In this section, we will illustrate the key-value store usage by walking through the store example which provides direct store access through USB serial.

We first define a helper to write a line to the USB serial.

fn writeln(buf: &[u8]) {
    serial::write_all(&UsbSerial, buf).unwrap();
    serial::write_all(&UsbSerial, b"\r\n").unwrap();
}

Because values may be at most 1023 bytes, there is a system to store large entries as multiple fragments of at most 1023 bytes each using multiple keys. To support those large entries, we define an abstract notion of keys. An abstract key is either exactly one key, or a contiguous range of keys.

enum Key {
    Exact(usize),
    Range(Range<usize>),
}

We then define helpers to dispatch to the regular or fragmented version based on the abstract key. The insert, find, and remove functions will be explained later.

fn insert(key: &Key, value: &[u8]) -> Result<(), Error> {
    match key {
        Key::Exact(key) => store::insert(*key, value),
        Key::Range(keys) => store::fragment::insert(keys.clone(), value),
    }
}

fn find(key: &Key) -> Result<Option<Box<[u8]>>, Error> {
    match key {
        Key::Exact(key) => store::find(*key),
        Key::Range(keys) => store::fragment::find(keys.clone()),
    }
}

fn remove(key: &Key) -> Result<(), Error> {
    match key {
        Key::Exact(key) => store::remove(*key),
        Key::Range(keys) => store::fragment::remove(keys.clone()),
    }
}

The first thing we do when the applet starts is print a short help describing how to use the applet.

fn main() {
    writeln(b"Usage: insert <key>[..<key>] <value>");
    writeln(b"Usage: find <key>[..<key>]");
    writeln(b"Usage: remove <key>[..<key>]");

We can then start the infinite loop processing exactly one command per iteration.

    loop {

We start a loop iteration by reading a command from the user. Only space, lower-case alphabetic characters, digits, and backspace are supported. We exit as soon as the user hits Enter.

        // Read the command.
        let mut command = String::new();
        loop {
            serial::write_all(&UsbSerial, format!("\r\x1b[K> {command}").as_bytes()).unwrap();
            match serial::read_byte(&UsbSerial).unwrap() {
                c @ (b' ' | b'.' | b'a' ..= b'z' | b'0' ..= b'9') => command.push(c as char),
                0x7f => drop(command.pop()),
                0x0d => break,
                _ => (),
            }
        }
        serial::write_all(&UsbSerial, b"\r\n").unwrap();

We then parse the command (described later). If the command is invalid, we print a message and continue to the next loop iteration.

        // Parse the command.
        let command = match Command::parse(&command) {
            Some(x) => x,
            None => {
                writeln(b"Failed: InvalidCommand");
                continue;
            }
        };

And we finally process the command (described later). If processing failed, we print a message with the error. Regardless of error, this is the end of the loop (and thus the main function) and we continue to the next iteration.

        // Process the command.
        if let Err(error) = command.process() {
            writeln(format!("Failed: {error:?}").as_bytes());
        }

To ease parsing and processing, we define a straightforward type for commands.

enum Command<'a> {
    Insert { key: Key, value: &'a str },
    Find { key: Key },
    Remove { key: Key },
}

The parsing function is also straightforward.

impl<'a> Command<'a> {
    fn parse(input: &'a str) -> Option<Self> {
        Some(match *input.split_whitespace().collect::<Vec<_>>().as_slice() {
            ["insert", key, value] => Command::Insert { key: Key::parse(key)?, value },
            ["find", key] => Command::Find { key: Key::parse(key)? },
            ["remove", key] => Command::Remove { key: Key::parse(key)? },
            _ => return None,
        })
    }

The process function is a Command method which may return a store error.

    fn process(&self) -> Result<(), Error> {

For insert commands, we simply forward to the store::insert() function (resp. store::fragment::insert() for fragmented entries) which maps a key (resp. a range of keys) to a value. If the key (resp. range of keys) was already mapped, it is overwritten. A key must be a number smaller than 4096. A range of keys must be non-empty. A value must be a slice of at most 1023 bytes (resp. 1023 bytes times the number of fragments).

        match self {
            Command::Insert { key, value } => insert(key, value.as_bytes()),

Remove commands are also straightforward. We use store::remove() which maps a key to nothing. It's not an error if the key wasn't mapped before.

            Command::Remove { key } => remove(key),

Finally, find commands are implemented using store::find() which takes a key and return the mapped value if any.

            Command::Find { key } => {
                match find(key)? {

We print a message if no value was found.

                    None => writeln(b"Not found."),

Otherwise, we try to convert the byte slice to a string slice. This should succeed for values that were inserted by this applet since we only accept alphanumeric characters. In that case, we simply print the value.

                    Some(value) => match core::str::from_utf8(&value) {
                        Ok(value) => writeln(format!("Found: {value}").as_bytes()),

However, because the store is persistent and keys are not yet partitioned by applets, we could read the values written by a previous applet for that key. And those values don't need to be valid UTF-8. In those cases, we print the value as a byte slice.

                        Err(_) => writeln(format!("Found (not UTF-8): {value:02x?}").as_bytes()),

The final code looks like this:

#![no_std]
wasefire::applet!();

use alloc::boxed::Box;
use alloc::format;
use alloc::string::String;
use alloc::vec::Vec;
use core::num::ParseIntError;
use core::ops::Range;
use core::str::FromStr;

use wasefire::usb::serial::UsbSerial;

fn main() {
    writeln(b"Usage: insert <key>[..<key>] <value>");
    writeln(b"Usage: find <key>[..<key>]");
    writeln(b"Usage: remove <key>[..<key>]");
    loop {
        // Read the command.
        let mut command = String::new();
        loop {
            serial::write_all(&UsbSerial, format!("\r\x1b[K> {command}").as_bytes()).unwrap();
            match serial::read_byte(&UsbSerial).unwrap() {
                c @ (b' ' | b'.' | b'a' ..= b'z' | b'0' ..= b'9') => command.push(c as char),
                0x7f => drop(command.pop()),
                0x0d => break,
                _ => (),
            }
        }
        serial::write_all(&UsbSerial, b"\r\n").unwrap();

        // Parse the command.
        let command = match Command::parse(&command) {
            Some(x) => x,
            None => {
                writeln(b"Failed: InvalidCommand");
                continue;
            }
        };

        // Process the command.
        if let Err(error) = command.process() {
            writeln(format!("Failed: {error:?}").as_bytes());
        }
    }
}

enum Command<'a> {
    Insert { key: Key, value: &'a str },
    Find { key: Key },
    Remove { key: Key },
}

impl<'a> Command<'a> {
    fn parse(input: &'a str) -> Option<Self> {
        Some(match *input.split_whitespace().collect::<Vec<_>>().as_slice() {
            ["insert", key, value] => Command::Insert { key: Key::parse(key)?, value },
            ["find", key] => Command::Find { key: Key::parse(key)? },
            ["remove", key] => Command::Remove { key: Key::parse(key)? },
            _ => return None,
        })
    }

    fn process(&self) -> Result<(), Error> {
        match self {
            Command::Insert { key, value } => insert(key, value.as_bytes()),
            Command::Find { key } => {
                match find(key)? {
                    None => writeln(b"Not found."),
                    Some(value) => match core::str::from_utf8(&value) {
                        Ok(value) => writeln(format!("Found: {value}").as_bytes()),
                        Err(_) => writeln(format!("Found (not UTF-8): {value:02x?}").as_bytes()),
                    },
                }
                Ok(())
            }
            Command::Remove { key } => remove(key),
        }
    }
}

enum Key {
    Exact(usize),
    Range(Range<usize>),
}

impl FromStr for Key {
    type Err = ParseIntError;

    fn from_str(s: &str) -> Result<Self, Self::Err> {
        match s.split_once("..") {
            Some((start, end)) => Ok(Key::Range(start.parse()? .. end.parse()?)),
            None => Ok(Key::Exact(s.parse()?)),
        }
    }
}

impl Key {
    fn parse(key: &str) -> Option<Self> {
        let key: Key = key.parse().ok()?;
        let valid = match &key {
            Key::Exact(key) => *key < 4096,
            Key::Range(keys) => !keys.is_empty() && keys.end < 4096,
        };
        if !valid {
            return None;
        }
        Some(key)
    }
}

fn insert(key: &Key, value: &[u8]) -> Result<(), Error> {
    match key {
        Key::Exact(key) => store::insert(*key, value),
        Key::Range(keys) => store::fragment::insert(keys.clone(), value),
    }
}

fn find(key: &Key) -> Result<Option<Box<[u8]>>, Error> {
    match key {
        Key::Exact(key) => store::find(*key),
        Key::Range(keys) => store::fragment::find(keys.clone()),
    }
}

fn remove(key: &Key) -> Result<(), Error> {
    match key {
        Key::Exact(key) => store::remove(*key),
        Key::Range(keys) => store::fragment::remove(keys.clone()),
    }
}

fn writeln(buf: &[u8]) {
    serial::write_all(&UsbSerial, buf).unwrap();
    serial::write_all(&UsbSerial, b"\r\n").unwrap();
}