Skip to main content
holden

debugging a zig tui with lldb

I am working through kilo, but using zig instead of c.

I have to redraw the terminal to show the editor, so print debugging won't work.

This is what I learned about using lldb.

build/run #

This example assumes the program is a single zig file.

Once the program is more complicated, use zig build.

// build
$ zig build-exe src/main.zig
// run
$ ./main

attach lldb #

Attach lldb to a program by name with -n or process ID with -p.

// attach to the process by name
$ lldb -n main

(lldb) process attach --name "main"
Process 165016 stopped
* thread #1, name = 'main', stop reason = signal SIGSTOP
    frame #0: 0x000000000103ec6b main`os.linux.x86_64.syscall3(number=read, arg1=0, arg2=140726740563076, arg3=1) at x86_64.zig:46:5
   43  	}
   44  	
   45  	pub fn syscall3(number: SYS, arg1: usize, arg2: usize, arg3: usize) usize {
-> 46  	   return asm volatile ("syscall"
   47  	       : [ret] "={rax}" (-> usize),
   48  	       : [number] "{rax}" (@intFromEnum(number)),
   49  	         [arg1] "{rdi}" (arg1),
Executable module set to "/home/h/kilo/main".

Now lldb is attached and showing the current state of the program:

  1. the process is stopped because of SIGSTOP
    • this is sent by the debugger when attaching
  2. before stopping, the program was on line 46 of x86_64.zig
  3. syscall3 was in the process of returning
    • the arguments reveal what is happening
      1. read: the read syscall with three arguments
      2. 0: the file descriptor for stdin
      3. 140726740563076: the memory address of the destination buffer
      4. 1: the number of bytes to read
    • the program was waiting for input on stdin

help #

Look at help to get your footing.

There is a section that lists aliases; these are the most commonly used commands.

help
(lldb) help
Debugger commands:
  apropos           -- List debugger commands related to a word or subject.
  breakpoint        -- Commands for operating on breakpoints (see 'help b' for
                       shorthand.)
  command           -- Commands for managing custom LLDB commands.
  diagnostics       -- Commands controlling LLDB diagnostics.
  disassemble       -- Disassemble specified instructions in the current
                       target.  Defaults to the current function for the
                       current thread and stack frame.
  dwim-print        -- Print a variable or expression.
  expression        -- Evaluate an expression on the current thread.  Displays
                       any returned value with LLDB's default formatting.
  frame             -- Commands for selecting and examing the current thread's
                       stack frames.
  gdb-remote        -- Connect to a process via remote GDB server.
                       If no host is specifed, localhost is assumed.
                       gdb-remote is an abbreviation for 'process connect
                       --plugin gdb-remote connect://:'
  gui               -- Switch into the curses based GUI mode.
  help              -- Show a list of all debugger commands, or give details
                       about a specific command.
  kdp-remote        -- Connect to a process via remote KDP server.
                       If no UDP port is specified, port 41139 is
                       assumed.
                       kdp-remote is an abbreviation for 'process connect
                       --plugin kdp-remote udp://:'
  language          -- Commands specific to a source language.
  log               -- Commands controlling LLDB internal logging.
  memory            -- Commands for operating on memory in the current target
                       process.
  platform          -- Commands to manage and create platforms.
  plugin            -- Commands for managing LLDB plugins.
  process           -- Commands for interacting with processes on the current
                       platform.
  quit              -- Quit the LLDB debugger.
  register          -- Commands to access registers for the current thread and
                       stack frame.
  script            -- Invoke the script interpreter with provided code and
                       display any results.  Start the interactive interpreter
                       if no code is supplied.
  session           -- Commands controlling LLDB session.
  settings          -- Commands for managing LLDB settings.
  source            -- Commands for examining source code described by debug
                       information for the current target process.
  statistics        -- Print statistics about a debugging session
  target            -- Commands for operating on debugger targets.
  thread            -- Commands for operating on one or more threads in the
                       current process.
  trace             -- Commands for loading and using processor trace
                       information.
  type              -- Commands for operating on the type system.
  version           -- Show the LLDB debugger version.
  watchpoint        -- Commands for operating on watchpoints.
Current command abbreviations (type 'help command alias' for more info):
  add-dsym  -- Add a debug symbol file to one of the target's current modules
               by specifying a path to a debug symbols file or by using the
               options to specify a module.
  attach    -- Attach to process by ID or name.
  b         -- Set a breakpoint using one of several shorthand formats.
  bt        -- Show the current thread's call stack.  Any numeric argument
               displays at most that many frames.  The argument 'all' displays
               all threads.  Use 'settings set frame-format' to customize the
               printing of individual frames and 'settings set thread-format'
               to customize the thread header.
  c         -- Continue execution of all threads in the current process.
  call      -- Evaluate an expression on the current thread.  Displays any
               returned value with LLDB's default formatting.
  continue  -- Continue execution of all threads in the current process.
  detach    -- Detach from the current target process.
  di        -- Disassemble specified instructions in the current target. 
               Defaults to the current function for the current thread and
               stack frame.
  dis       -- Disassemble specified instructions in the current target. 
               Defaults to the current function for the current thread and
               stack frame.
  display   -- Evaluate an expression at every stop (see 'help target
               stop-hook'.)
  down      -- Select a newer stack frame.  Defaults to moving one frame, a
               numeric argument can specify an arbitrary number.
  env       -- Shorthand for viewing and setting environment variables.
  exit      -- Quit the LLDB debugger.
  f         -- Select the current stack frame by index from within the current
               thread (see 'thread backtrace'.)
  file      -- Create a target using the argument as the main executable.
  finish    -- Finish executing the current stack frame and stop after
               returning.  Defaults to current thread unless specified.
  h         -- Show a list of all debugger commands, or give details about a
               specific command.
  history   -- Dump the history of commands in this session.
               Commands in the history list can be run again using "!". 
               "!-" will re-run the command that is  commands
               from the end of the list (counting the current command).
  image     -- Commands for accessing information for one or more target
               modules.
  j         -- Set the program counter to a new address.
  jump      -- Set the program counter to a new address.
  kill      -- Terminate the current target process.
  l         -- List relevant source code using one of several shorthand formats.
  list      -- List relevant source code using one of several shorthand formats.
  n         -- Source level single step, stepping over calls.  Defaults to
               current thread unless specified.
  next      -- Source level single step, stepping over calls.  Defaults to
               current thread unless specified.
  nexti     -- Instruction level single step, stepping over calls.  Defaults to
               current thread unless specified.
  ni        -- Instruction level single step, stepping over calls.  Defaults to
               current thread unless specified.
  p         -- Print a variable or expression.
  parray    -- parray   -- lldb will evaluate EXPRESSION to
               get a typed-pointer-to-an-array in memory, and will display
               COUNT elements of that type from the array.
  po        -- Evaluate an expression on the current thread.  Displays any
               returned value with formatting controlled by the type's author.
  poarray   -- poarray   -- lldb will evaluate EXPRESSION to
               get the address of an array of COUNT objects in memory, and will
               call po on them.
  print     -- Print a variable or expression.
  q         -- Quit the LLDB debugger.
  r         -- Launch the executable in the debugger.
  rbreak    -- Sets a breakpoint or set of breakpoints in the executable.
  re        -- Commands to access registers for the current thread and stack
               frame.
  run       -- Launch the executable in the debugger.
  s         -- Source level single step, stepping into calls.  Defaults to
               current thread unless specified.
  shell     -- Run a shell command on the host.
  si        -- Instruction level single step, stepping into calls.  Defaults to
               current thread unless specified.
  sif       -- Step through the current block, stopping if you step directly
               into a function whose name matches the TargetFunctionName.
  step      -- Source level single step, stepping into calls.  Defaults to
               current thread unless specified.
  stepi     -- Instruction level single step, stepping into calls.  Defaults to
               current thread unless specified.
  t         -- Change the currently selected thread.
  tbreak    -- Set a one-shot breakpoint using one of several shorthand formats.
  undisplay -- Stop displaying expression at every stop (specified by stop-hook
               index.)
  up        -- Select an older stack frame.  Defaults to moving one frame, a
               numeric argument can specify an arbitrary number.
  v         -- Show variables for the current stack frame. Defaults to all
               arguments and local variables in scope. Names of argument,
               local, file static and file global variables can be specified.
  var       -- Show variables for the current stack frame. Defaults to all
               arguments and local variables in scope. Names of argument,
               local, file static and file global variables can be specified.
  vo        -- Show variables for the current stack frame. Defaults to all
               arguments and local variables in scope. Names of argument,
               local, file static and file global variables can be specified.
  x         -- Read from the memory of the current target process.
For more information on any command, type 'help '.

call stack #

bt shows the call stack.

(lldb) bt
* thread #1, name = 'kilo', stop reason = signal SIGSTOP
  * frame #0: 0x0000000001044d7b kilo`os.linux.x86_64.syscall3(number=read, arg1=0, arg2=140731972458060, arg3=1) at x86_64.zig:46:5
    frame #1: 0x000000000103a195 kilo`os.linux.read(fd=0, buf="", count=1) at linux.zig:829:20
    frame #2: 0x0000000001039cfe kilo`main.Editor.readKey(self=0x00007ffeb73a15f0) at main.zig:56:30
    frame #3: 0x000000000103b51e kilo`main.Editor.processKeypress(self=0x00007ffeb73a15f0) at main.zig:104:41
    frame #4: 0x000000000103bef9 kilo`main.main at main.zig:306:35
    frame #5: 0x0000000001038956 kilo`start.posixCallMainAndExit [inlined] start.callMain at start.zig:524:37
    frame #6: 0x000000000103894a kilo`start.posixCallMainAndExit [inlined] start.callMainWithArgs at start.zig:482:20
    frame #7: 0x00000000010388ed kilo`start.posixCallMainAndExit at start.zig:438:36
    frame #8: 0x0000000001038472 kilo`start._start at start.zig:266:5

This shows where the process was when it froze. processKeypress shows why the program was waiting for input.

v shows variables in the current stack frame.

(lldb) v
(os.linux.syscalls.X64) number = read
(unsigned long) arg1 = 0
(unsigned long) arg2 = 140731972458060
(unsigned long) arg3 = 1

Look at a type with type lookup.

(lldb) type lookup os.linux.syscalls.X64
enum os.linux.syscalls.X64 {
    read,
    write,
    ...
}

This is every syscall zig knows about.

breakpoints #

I want to move into my code.

Set a breakpoint with b.

(lldb) b kilo`main.Editor.readKey
Breakpoint 1: where = kilo`main.Editor.readKey + 39 at main.zig:52:33, address = 0x0000000001039c77

NOTE: b takes a regex, so b readKey would do the same thing.

This is a function from the earlier call stack, but you can search for symbols.

(lldb) image lookup -r -s open
11 symbols match the regular expression 'open' in /home/h/kilo/main:
        Address: main[0x0000000001038fa0] (main.PT_LOAD[1]..text + 2880)
        Summary: main`main.Editor.open at main.zig:121
        Address: main[0x0000000001075a40] (main.PT_LOAD[1]..text + 251360)
        Summary: main`posix.openatZ at posix.zig:1742
        Address: main[0x0000000001045ca0] (main.PT_LOAD[1]..text + 55360)
        Summary: main`debug.openSelfDebugInfo at debug.zig:1046
        Address: main[0x0000000001055570] (main.PT_LOAD[1]..text + 119056)
        Summary: main`dwarf.openDwarfDebugInfo at dwarf.zig:2175
        Address: main[0x0000000001058ec0] (main.PT_LOAD[1]..text + 133728)
        Summary: main`fs.Dir.openFile at Dir.zig:798
        Address: main[0x0000000001059010] (main.PT_LOAD[1]..text + 134064)
        Summary: main`fs.openSelfExe at fs.zig:494
        Address: main[0x0000000001079100] (main.PT_LOAD[1]..text + 265376)
        Summary: main`fs.Dir.openFileZ at Dir.zig:831
        Address: main[0x00000000010794b0] (main.PT_LOAD[1]..text + 266320)
        Summary: main`fs.openFileAbsoluteZ at fs.zig:310
        Address: main[0x0000000001086fd0] (main.PT_LOAD[1]..text + 322416)
        Summary: main`os.linux.openat at linux.zig:1113
        Address: main[0x000000000108bb60] (main.PT_LOAD[1]..text + 341760)
        Summary: main`posix.openZ at posix.zig:1573
        Address: main[0x00000000010a3190] (main.PT_LOAD[1]..text + 437552)
        Summary: main`os.linux.open at linux.zig:1095

b without an argument lists all your breakpoints.

(lldb) b
Current breakpoints:
1: name = 'main.Editor.readKey', module = kilo, locations = 1, resolved = 1, hit count = 1
  1.1: where = kilo`main.Editor.readKey + 39 at main.zig:52:33, address = 0x0000000001039c77, resolved, hit count = 1

controlling execution #

The program is still stopped, so continue.

(lldb) continue
Process 165016 resuming

Trigger the new breakpoint, and return to lldb.

The output will be similar to when your first attached, but the stop reason will be the new breakpoint.

Process 564884 stopped
* thread #1, name = 'kilo', stop reason = breakpoint 1.1
    frame #0: 0x0000000001039c77 kilo`main.Editor.readKey(self=0x000000004d430001) at main.zig:52:33
   49  	       self.rows.deinit();
   50  	   }
   51  	
-> 52  	   fn readKey(self: *Self) !u8 {
   53  	       var seq = try self.allocator.alloc(u8, 3);
   54  	       defer self.allocator.free(seq);
   55  	

Notice the arrow pointing to the current line.

n will move over the next line, s will move into.

   foo()
-> bar()
   baz()

Over would mean move to baz()

   foo()
   bar()
-> baz()

Into would mean move "into" bar().

-> fn bar() {
       ...
   }

finish will execute until the function returns.

NOTE: s followed by finish is the same as n.

looking at memory #

p prints.

(lldb) p self
(main.Editor *) 0x00007fff56e0d030

Note that self is a pointer (*).

Read the memory at a location with x.

(lldb) x self
0x7fff56e0d030: b2 fa e0 56 ff 7f 00 00 07 00 00 00 00 00 00 00  ...V............
0x7fff56e0d040: c8 42 0f 01 00 00 00 00 c8 41 00 01 00 00 00 00  .B.......A......

I want to find the value of self.c, so I print it with p.

NOTE: self.c is automatically converted to self->c, because self is a pointer.

(lldb) p self->c
(unsigned char[1]) "\U0000001b"

watchpoints #

I need to know when self.rows is updated.

However, self.rows is an ArrayList, and watchpoints can only watch a pointers worth of memory (usize) at once.

(lldb) watchpoint set variable self->rows.items.len
Watchpoint created: Watchpoint 3: addr = 0x7fff56e0d068 size = 8 state = enabled type = m
    declare @ '/home/h/kilo/src/main.zig:111'
    watchpoint spec = 'self->rows.items.len'
Watchpoint 3 hit:
    
new value: 1

You can also add commands which run when the watchpoint triggers.

// show call stack every time the watchpoint triggers
(lldb) watchpoint command add 1

Enter your debugger command(s).  Type 'DONE' to end.

> bt

> DONE

gui #

gui will open an interface with better visualization.

I like it for viewing the code around the breakpoint.

| LLDB (F1) | Target (F2) | Process (F3) | Thread (F4) | View (F5) | Help (F6) |                                                                   
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐┌───────────────────┐
│ kilo`array_list.ArrayListAligned(main.Row,null).addManyAtAssumeCapacity                                           ││ ◆─process 721751           │
│  210 │         /// operations.                                                                                    ││ └─◆─thread #1: tid = 0xb035│
│  211 │         /// Asserts that there is enough capacity for the new elements.                                    ││   ├─#0: array_list.ArrayLis│
│  212 │         /// Invalidates pre-existing pointers to elements at and after `index`, but                        ││   ├─#1: array_list.ArrayLis│
│  213 │         /// does not invalidate any before that.                                                           ││   ├─#2: array_list.ArrayLis│
│  214 │         /// Asserts that the index is in bounds or equal to the length.                                    ││   ├─#3: main.Editor.insertR│
│  215 │         pub fn addManyAtAssumeCapacity(self: *Self, index: usize, count: usize) []T {                      ││   ├─#4: main.Editor.open + │
│  216 │             const new_len = self.items.len + count;                                                        ││   ├─#5: main.main + 1291   │
│  217 │             assert(self.capacity >= new_len);                                                              ││   ├─#6: start.posixCallMain│
│  218 │             const to_move = self.items[index..];                                                           ││   ├─#7: start.posixCallMain│
│  219 │             self.items.len = new_len;                                                                      ││   ├─#8: start.posixCallMain│
│  220 │◆            mem.copyBackwards(T, self.items[index + count ..], to_move);         <<< Thread 1: watchpoint 3││   └─#9: start._start + 18  │
│  221 │             const result = self.items[index..][0..count];                                                  ││                            │
│  222 │             @memset(result, undefined);                                                                    ││                            │
│  223 │             return result;                                                                                 ││                            │
│  224 │         }                                                                                                  ││                            │
│  225 │                                                                                                            ││                            │
│  226 │         /// Insert slice `items` at index `i` by moving `list[i .. list.len]` to make room.                ││                            │
│  227 │         /// This operation is O(N).                                                                        ││                            │
│  228 │         /// Invalidates pre-existing pointers to elements at and after `index`.                            ││                            │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘│                            │
┌────────────────────────────────────────────────────────────────────────────────────────────────────────┐│                            │
│ ◆─(*array_list.ArrayListAligned(main.Row,null)) self = 0x00007fff56e0d060                                         ││                            │
│ (usize) index = 2                                                                                                 ││                            │
│ (usize) count = 1                                                                                                 ││                            │
│ (usize) new_len = 3                                                                                               ││                            │
│ ◆─([]main.Row) to_move                                                                                            ││                            │
│ ◆─([]main.Row) result                                                                                             ││                            │
│ (u8) dwarf.call_frame.Opcode.lo_inline = '@'                                                                      ││                            │
│ (usize) heap.general_purpose_allocator.default_test_stack_trace_frames = 6                                        ││                            │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘└────────────────────────────┘
Process: 721751    stopped              Thread: 721751      Frame:   0  PC = 0x0000000001076ee2