debugger backtrace tests, fixes, ordering

[bburdette]

Motivation

The debugger got a little bit broken from PR 6204. Also, roberth requested some testing. This PR fixes a few debugger problems and adds a simple test for backtrace sequence.

This is by no means a complete test suite for the debugger. But its a start.

What was changed:

• pos was no longer showing for primops, most importantly builtins.break.
• replaced hardcoded error: with printing the verbosity of an error in the backtrace. this is nice because builtins.break shows as info: instead of error:.
• reversed order of backtraces when printed, to match the order of --stack-trace output.
• reversed order of :env output to match the backtrace output.
• there was an extra backtrace being added in by PR 6204, which was storing some location data in a DebugTrace for its own purposes. That backtrace looked like this:
  
  I changed that code to use a new datastructure 'WithFrame' to store location outside the DebugTrace stack. (removed altogether now)
• added tests/debugger-backtrace.sh. This has a few simple nix files and evals them to check backtrace sequence against expected outputs.

[bburdette]

Ok I think this is ready for review. I removed the 'withTrace' code and didn't see a difference from that in the lang test output.

We gained some additional stack locations in the lang test traces because of primop pos being available again.

[lorenzleutgeb]

@bburdette have you considered using expect to script interaction with the debugger?

[bburdette]

I looked at expect for about an hour or so. Hard to tell how to use it for this case. I patterned the testRepl function in this after similar code in repl.sh. If expect is a requirement to land this I can delve deeper I suppose.

One aspect here is the output from the repl includes some hex addresses from the nix program, which will vary depending on the build. So you can't count on a specific output being emitted, it has to be filtered.

[lorenzleutgeb]

Hard to tell how to use it for this case.

Well, the script would spawn Nix to evaluate a file that contains builtins.break using the --debugger argument, and then it would interact by executing various commands.

If expect is a requirement to land this I can delve deeper I suppose.

I am just interested in the debugger, not a Nix maintainer, so I am not in a position to define requirements. I just genuinely think that expect would be a good fit.

One aspect here is the output from the repl includes some hex addresses from the nix program, which will vary depending on the build. So you can't count on a specific output being emitted, it has to be filtered.

Yeah, you can match output with regular expressions, and you can also skip matching output when it's not required. See https://stackoverflow.com/a/37255077

It's late in my timezone now, but I could write one of your tests as an expect script tomorrow (in 12-20 hours) so that you have a concrete example. Do you think that would help you?

[bburdette]

It seems like expect must not be used elsewhere in the nix tests, since they have their own expect/expectstderr functions in vars-and-functions.sh. That might need to be addressed.

If you feel like making an example I'd be interested to see it. On that expect documentation page you referenced, most of the links are broken, particularly to the book and for 'expect for unix'. Do you have links for those, or some other more systematic introduction to expect/tcl besides just the man page?

[lorenzleutgeb]

If you feel like making an example I'd be interested to see it.

#! /usr/bin/env -S nix shell nixpkgs#expect --command expect

spawn nix eval --debugger --file debugger/debugger-attr-bad.nix

# NOTE: To tell Nix to not output any color, try this.
# However, REPL output will still be colored, so it's not
# as useful...
#spawn env NO_COLOR=true nix eval --debugger --file debugger/debugger-attr-bad.nix

expect "Starting REPL"

# NOTE: We wait for the prompt of the REPL,
# and then send a command.
expect "nix-repl> "
send ":bt\n"

# NOTE: Using regex matching here, because the ANSI escape codes
# for colors in the output must be matched, and I am too lazy
# to encode the colors here...
expect -re {0.*error:.*cannot add.*a string.*to an integer}

# NOTE: Here, you could add expect calls to check that other
# stack frames are mentioned.

expect "nix-repl> "
send ":q\n"

expect "… while evaluating the file"

On that expect documentation page you referenced, most of the links are broken, particularly to the book and for 'expect for unix'. Do you have links for those, or some other more systematic introduction to expect/tcl besides just the man page?

There are lots of tutorials, I can't recommend any one in particular. Maybe start with those at https://wiki.tcl-lang.org/page/Expect+Tutorials

Maybe using pexpect which is an expect clone written in Python would be a better idea for maintainability. Some feedback from maintainers regarding expect/pexpect and the question

It seems like expect must not be used elsewhere in the nix tests, since they have their own expect/expectstderr functions in vars-and-functions.sh. That might need to be addressed.

would be helpful.

[bburdette]

That's somewhat similar to what I ended up with, though I'm still hazy on some aspects. Time being limited, my inclination right now is to get this in as-is if possible, and take a look at expect/pexpect etc for a separate PR. I agree it does seem like a good fit for the interactive features of the debugger.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2946

[lf-]

That's somewhat similar to what I ended up with, though I'm still hazy on some aspects. Time being limited, my inclination right now is to get this in as-is if possible, and take a look at expect/pexpect etc for a separate PR. I agree it does seem like a good fit for the interactive features of the debugger.

If you've not got around to this yet, I have looked into expect(1) and it's a horrible horrible mess of tcl. I'm going to implement something relatively akin to this in another PR: https://bitheap.org/cram/

The idea is that you can essentially write a test that is an annotated transcript of a repl session.

[lorenzleutgeb]

@lf- is there any previous discussion regarding the use of cram? It was last released on 2016-02-24, compared to pexpect which was last released on 2023-11-05.

[lf-]

@lf- is there any previous discussion regarding the use of cram? It was last released on 2016-02-24, compared to pexpect which was last released on 2023-11-05.

I'm not using cram, I'm writing something very similar in C++ integrated with Nix.