Stivi Guranjaku - 29-Sept-26
This is my first post in a series about programming small projects in Zig. Zig has been on my list of languages to try out for a few months and I've finally stopped messing around with random side projects to start learning Zig.
Many of you are probably familiar with the tool xxd. It's part of the Vim source code that has been bundled as a standalone command line tool.
It's a very useful utility and relatively simple, which is why I thought recreating it in zig would be a good introduction to the language.
To keep it simple, I will only be reading from stdin and writing to stdout. With that out of the way, let's start programming!
pub fn main(init: std.process.Init) !void {
const io = init.io;
var stdinbuf: [16 * 1024]u8 = undefined;
var stdoutbuf: [16 * 1024]u8 = undefined;
var stdin = std.Io.File.stdin().readerStreaming(io, &stdinbuf);
var stdout = std.Io.File.stdout().writerStreaming(io, &stdoutbuf);
var offset: usize = 0;
while (true) {
const row = stdin.interface.peek(16) catch |err| switch (err) {
error.EndOfStream => {
const tail = stdin.interface.buffered();
if (tail.len != 0) {
try printRow(&stdout.interface, offset, tail);
_ = try stdin.interface.discard(.limited(tail.len));
}
break;
},
else => return err,
};
if (row.len == 0) break;
try printRow(&stdout.interface, offset, row);
offset += row.len;
_ = try stdin.interface.discard(.limited(row.len));
}
try stdout.interface.flush();
}
I came from a C background, and at first found it kind of difficult to read/write from
stdin/stdout. However Zig has a very powerful design paradigm: there is no hidden copying or
allocations, everything is done with intention. Let's look at how we read from stdin (the logic is
similar for writing to stdout).
First, we need to get a file handle to stdin. With Zig, this is done by getting the identifier for
stdin with std.Io.File.stdin(), and then initializing a Reader struct, passing a scratch buffer
to stream into (since we are setting it to streaming mode) and the init.io interface, which is
what actually routes to the relevant syscall for reading.
After reading whatever we streamed into that buffer, we have to explicitly discard whatever is in
there.
In C, a lot of this is abstracted in scanf and printf (unless you explicitly ask for these
functions not to do the flushing for you). So Zig actually gives you a bit more information on
what's happening behind the scenes.
Now Zig does have builtin helpers for I/O where you can pass custom iterators, but I haven't gotten
that far yet and implemnted it "manually" here via peek and discard. I'm sure there's a better
way to write this.
Now that we understand how to read and write, let's get onto the business logic of the application, parsing the bytes we recieve:
const hex = "0123456789abcdef";
const row_width = 78;
const bytes_per_row = 16;
fn printRow(interface: *std.Io.Writer, offset: usize, row: []const u8) !void {
var printbuf: [row_width]u8 = undefined;
const off: u32 = @intCast(offset);
var pos: usize = 0;
for (0..8) |i| {
const shift: u5 = @intCast((7 - i) * 4);
printbuf[pos] = hex[(off >> shift) & 0x0f];
pos += 1;
}
printbuf[pos] = ':';
printbuf[pos + 1] = ' ';
pos += 2;
for (0..bytes_per_row) |i| {
if (i < row.len) {
const byte = row[i];
printbuf[pos] = hex[byte >> 4];
printbuf[pos + 1] = hex[byte & 0x0f];
} else {
printbuf[pos] = ' ';
printbuf[pos + 1] = ' ';
}
printbuf[pos + 2] = ' ';
pos += 3;
}
printbuf[pos] = '|';
pos += 1;
for (0..bytes_per_row) |i| {
const byte = if (i < row.len) row[i] else null;
printbuf[pos] = if (byte) |b|
if (b >= 0x20 and b <= 0x7e) b else '.'
else
' ';
pos += 1;
}
printbuf[pos] = '|';
printbuf[pos + 1] = '\n';
pos += 2;
try interface.writeAll(printbuf[0..pos]);
}
First of all, Zig has builtins for formatting which could have been used instead, but this is a for
fun project so I thought I would implement it myself. If you haven't converted bytes to hex before,
each byte can be split into two $4$ bit components: the high nibble and the low nibble. Let's take
01011101 as an example. The high nibble here is 0101. we can isolate this by performing a bit
shift to the right by $4$. 0101 = 5, so in our lookup table this gets index $5$, which is $5$.
& 0x0f takes the lower four bytes by performing a bitwise and of 01011101 and 00001111. We can
use this for finding the per row offset as well. In xxd, the offset is a $32$-bit integer. By
shifting the offset right by $(7 - i) * 4$ and masking it with 0x0f, we isolate each $4$-bit
nibble from left to right. We use that nibble as an index into our hex array to pull out the correct
character (0 through f). The rest of the function is obvious, minus writeAll. It guarantees
that every single byte you pass into it gets written by running a while loop that calls
Writer.write to push data into the writer's internal buffer by either memcpy if the space
permits or draining to handle the overflow chunk. Because a single write call may only consume a
portion of the input (think that the writer's buffer is almost full and whatever you may write would
result in an overflow), writeAll adds the number of successfully written bytes to its index
tracker and loops until all bytes are written.
I'm sure this implementation can be improved and interated on but I am eager to learn more Zig and had a blast learning the language so far. If you are also trying to learn Zig I would suggest completing a few of the Ziglings exercices until you feel comfortable recreating small coreutils or easy/medium Leetcodes. The standard library documentation is quite good, and the source code is even better. Using zls to jump to the actual implementations has given me a good insight on how to use some of the interfaces.
I don't think Zig will replace C for existing programs, and don't forsee wide adoption until a stable release. I do think it will become popular amongst developers who like to understand internals and systems programming. I'm really looking forward to Zig's further development and I think I will be using Zig for as much as I can moving forward.
After spending a little more time with Zig's I/O interfaces, I simplified the implementation. Here's the updated code, with comments explaining the changes:
const std = @import("std");
const hex = "0123456789abcdef";
const bytes_per_row = 16;
const row_width = 10 + bytes_per_row * 3 + 1 + bytes_per_row + 2;
// Accept the displayed offset as a u32 directly; the caller now does the cast.
fn printRow(interface: *std.Io.Writer, offset: u32, row: []const u8) !void {
var printbuf: [row_width]u8 = undefined;
var pos: usize = 0;
for (0..8) |i| {
const shift: u5 = @intCast((7 - i) * 4);
// Use offset directly instead of a separate cast variable.
printbuf[pos] = hex[(offset >> shift) & 0x0f];
pos += 1;
}
printbuf[pos] = ':';
printbuf[pos + 1] = ' ';
pos += 2;
for (0..bytes_per_row) |i| {
if (i < row.len) {
const byte = row[i];
printbuf[pos] = hex[byte >> 4];
// Correction to the explanation above: this masks the lower four
// bits (the low nibble), not the lower four bytes.
printbuf[pos + 1] = hex[byte & 0x0f];
} else {
printbuf[pos] = ' ';
printbuf[pos + 1] = ' ';
}
printbuf[pos + 2] = ' ';
pos += 3;
}
printbuf[pos] = '|';
pos += 1;
for (0..bytes_per_row) |i| {
const byte = if (i < row.len) row[i] else null;
printbuf[pos] = if (byte) |b|
if (b >= 0x20 and b <= 0x7e) b else '.'
else
' ';
pos += 1;
}
printbuf[pos] = '|';
printbuf[pos + 1] = '\n';
pos += 2;
try interface.writeAll(printbuf[0..pos]);
}
pub fn main(init: std.process.Init) !void {
const io = init.io;
var stdinbuf: [16 * 1024]u8 = undefined;
var stdoutbuf: [16 * 1024]u8 = undefined;
var stdin = std.Io.File.stdin().readerStreaming(io, &stdinbuf);
var stdout = std.Io.File.stdout().writerStreaming(io, &stdoutbuf);
var offset: usize = 0;
// Read into a separate row buffer, sized using the same constant as printRow.
var row: [bytes_per_row]u8 = undefined;
while (true) {
// Fill the row, or return fewer bytes at EOF. This consumes the input,
// replacing peek, discard, and the special EndOfStream tail handling.
const len = try stdin.interface.readSliceShort(&row);
if (len == 0) break;
// Pass only the bytes read so the final partial row is padded correctly.
// The cast moved here, but offsets still have to fit in 32 bits
try printRow(&stdout.interface, @intCast(offset), row[0..len]);
offset += len;
}
try stdout.interface.flush();
}