After setting up the ESP32 toolchain and confirming regular GDB debugging is working as described in my previous post, now it is time to configure VSCode for remote debugging of the ESP32.
I've also added Part 3 - using ESP WIP OpenOCD.
Everything is pretty straightforward here, well except for the details of actually debugging in VSCode. I had quite an ordeal with that, as described in github issue #763: Remote Debugging of ESP32 causes EXCEPTION: System.ArgumentOutOfRangeException.
In the end, it seems that somehow VSCode was confused on the initial *state* of the ESP32 when debugging is first initiated. The first MI commands would get the ESP32 into some sort of weird state "internal to GDB":
C next: {"threadId":-2}
1: (812785) <-1022-exec-next
1: (812793) ->1022^running
1: (812793) ->*running,thread-id="all"
1: (812793) ->(gdb)
1: (812793) 1022: elapsed time 8
1: (812794) ->(gdb)
R: {"success":true,"message":null,"request_seq":11,"command":"next","body":null,"running":false,"refs":null,"seq":0,"type":"response"}
1: (812903) ->@"esp32.cpu0: Target halted, pc=0x40080E77\n"
E output: {"category":"stdout","output":"@\"esp32.cpu0: Target halted, pc=0x40080E77\\n\"\n","data":null,"type":"output"}
@"esp32.cpu0: Target halted, pc=0x40080E77\n"
1: (812932) ->~"/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.\nA problem internal to GDB has been detected,\nfurther debugging may prove unreliable.\nQuit this debugging session? "
1: (812932) ->~"(y or n) [answered Y; input not from terminal]\n"
E output: {"category":"stdout","output":"/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.\nA problem internal to GDB has been detected,\nfurther debugging may prove unreliable.\nQuit this debugging session? ","data":null,"type":"output"}
1: (812932) ->&"\nThis is a bug, please report it."
/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? I opened an espressif issue on giuthub for this one: [TW#12876] VSCode Debugging via MI causes internal GDB error #645. Fortunately - with the help of @pieandcakes - I was able to determine a (yes, somewhat wonky) sequence of steps to persuade VSCode to debug the ESP32!
I have my openocd running on Ubuntu with samba, and have mapped my windows Z: drive to the root of the ubuntu file system. For that, my smb.conf looks like this:
## edit the file /etc/samba/smb.conf and put in these lines at the end (without single # comment markers!)
##*******************************************************************************************************
[c$]
comment= root
path=/
browseable=Yes
writeable=Yes
only guest=no
create mask=0777
directory mask=0777
public=no
## also, make sure the authentication line is not commented out
####### Authentication #######
# security = user
##*******************************************************************************************************
# try to keep away WannaCry
[global]
min protocol = SMB2
First, browse to the Hello World "Build" directory in Windows File Explorer. Right click and click "Open with Code". The select Debug - Open Configurations. This should create a file called launch.json - in my case, in:
Z:\home\gojimmypi\esp\hello_world\build\.vscode\
Here's my launch.json for debugging the ESP32:
{
"version": "0.2.0",
"configurations": [
{
"name": "OpenOCD Debug",
"type": "cppdbg",
"request": "launch",
"miDebuggerPath": "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe",
"program": "Z:/home/gojimmypi/esp/hello_world/build/hello-world.elf",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
},
{
"text": "file 'Z:/home/gojimmypi/esp/hello_world/build/hello-world.elf'"
},
{
"text": "target remote 192.168.174.129:3333"
},
{
"text": "monitor reset init"
}
],
"externalConsole": false,
"cwd": "Z:/home/gojimmypi/esp/hello_world/build/",
"logging": {
"trace": true,
"traceResponse": true,
"engineLogging": true
}
}
]
}
Note that you need to choose between "miDebuggerServerAddress": "192.168.174.129:3333"
and "text": "target remote 192.168.174.129:3333"
- but not both! (see that first question in github issue 763). You will, of course - need to edit the IP address to point to your own OpenOCD server, otherwise you will see this error in VSCode:Unable to start debugging. Unexpected GDB output from command "-interpreter-exec console "target remote 192.168.174.129:3333""If you don't have a VM or a Linux box, perhaps consider a Raspberry Pi. :)
The line
"logging": {
"trace": true,
"traceResponse": true,
"engineLogging": trueis optional, but tremendously helpful in actually seeing what's going on between VSCode and the ESP32.
I created a few gists that can either be put into script files, or copy/pasted for some basic operations. I run each of these in a separate putty shell connection. First, the compile & flash:
#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf
sudo chmod 777 /dev/ttyUSB0
cd ~/esp/hello_world
make flash
In this same "compile window", it can be helpful to run the serial monitor. I like keeping it in the same window as make/building, so that I am sure to stop the monitor when reflashing.
make monitor
Next, run openocd server in yet another putty connection (when successful, the openocd will not exit, rather it continues running as a server for your VSCode GDB connection as a foreground task):
#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf
cd ~/workspace/openocd-esp32/tcl
sudo openocd -f interface/ftdi/olimex-arm-usb-ocd-h-1MHz.cfg -c "transport select jtag" -f target/ESP32-RTOS-none.cfg
A successful launch of openocd will look something like this:
0 $ sudo openocd -f interface/ftdi/olimex-arm-usb-ocd-h-1MHz.cfg -c "transport select jtag" -f target/ESP32-RTOS-none.cfg
[sudo] password for gojimmypi:
Open On-Chip Debugger 0.10.0-dev-g372bb59 (2017-05-19-10:35)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
adapter speed: 1000 kHz
jtag
force hard breakpoints
Info : clock speed 1000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : esp32.cpu0: Target halted, pc=0x400D1158
then the xtensa-esp32-elf-gdb debug client (the find is of course optional) this one with gdb commands in hello_world,gdb:
#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf
#find /. -name xtensa-esp32-elf-gdb 2>/dev/null
cd ~/esp/hello_world/build
xtensa-esp32-elf-gdb -tui -x hello_world.gdb hello-world.elf
Alternatively launch the xtensa gdb client simply like this (assume paths properly set as shown above):
xtensa-esp32-elf-gdb hello-world.elf
A successful launch of gdb will look something like this:
gojimmypi@ubuntu : ~/esp/hello_world/build
0 $ xtensa-esp32-elf-gdb hello-world.elf
GNU gdb (crosstool-NG crosstool-ng-1.22.0-61-gab8375a) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from hello-world.elf...done.
(gdb)
target remote localhost:3333
Remote debugging using localhost:3333
0x400d1158 in esp_vApplicationIdleHook () at /home/gojimmypi/esp/esp-idf/components/esp32/./freertos_hooks.c:52
52 asm("waiti 0");
(gdb)
We just wanted to confirm OpenOCD was working and that we could connect with linux GDB, so exit/quit.
The Windows 10 executable gdb is needed to debug in VSCode on Windows, so as shown on: http://esp-idf.readthedocs.io/en/latest/get-started/windows-setup.html it can be downloaded from here
(mine took HOURS to download)
When ready to start debugging in Windows, locate your xtensa-esp32-elf-gdb.exe, Mine is located in:
C:\msys32\opt\xtensa-esp32-elf\bin
Note the line in the VSCode launch.json:
"miDebuggerPath": "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe",
That's the one that calls the Windows GDB client to talk to the Ubuntu OpenOCD server. This gives us the best of both worlds: assimilation! Here we can have all the reliable compile of ESP32 code in linux, yet the ability to edit and debug from Windows!
Pay attention to the forward and back-slashes in the launch.json file. See vscode-cpptools issue #706.
The details of these steps are shown in my github issue (in particular the 8th comment after the question) if you want to see more details, but in summary:
Be sure the OpenOCD server is running as described above. Press F5 to start debugging. You should see "Exception has occurred.Trace/breakpoint trap." It appears we are debugging, but we are not. Target (appears to be) running. VSCode thinks it is paused.
If you try to single step here, all bets are off; xtensa GDB crashes, VSCode debugging stops. Game over.
If instead VSCode immediately stops debugging and you see something in the debug log like:
E output: {"category":"stderr","output":"ERROR: Unable to start debugging. Failed to find thread 1 for break event\r\n","data":null,"type":"output"}
ERROR: Unable to start debugging. Failed to find thread 1 for break eventOtherwise if all is still going according to plan: type
-exec continue at the VSCode > prompt.
Observe debug output. Note that despite -exec continue we are now actually in halt state. One of the lines should say:
(16086) ->@"esp32.cpu0: Target halted, pc=0x400F4AB3\n"Press the VSCode Play button (F5). Observe error:
ERROR: Cannot execute this command while the selected thread is running.
Only after the above steps are completed, now press the pause button. Observe output. There should be no errors.
Press a code-stepping button, such as Shift-F11.... tada! Single step debugging!
For an even better experience - see FHFS comment on espressif/esp-idf issue 303,
"if esp-idf is in the project folder symbols get found automatically. for windows you can also add the msys2, mingw32 and xtensa-esp32-elf bin folders to the path in tasks.json. make monitor has some trouble starting a console, you can edit the idf_monitor.py to output to stderr(task output window), or make a .cmd file with the python command in it and run that command from tasks.
You can see how I work with it esp-idf-VSCode-template"
I'm still working on polishing these instructions. Hopefully you've found this useful. Feedback is welcome and appreciated. Send me a message on twitter, github pull request or gmail.
Various tips:
GDB tip: Ctrl-x Ctrl-a in gdb to enter TUI mode.
You can press the EN button on the ESP32; it acts like reset and resolves many problems without having to power cycle (which will likely also need permissions re-applied,etc)
Note that as of May 2017, there's an issue in VSCode regarding the direction and quantity of path slashes as note in gitbub vscode-cpptools issue #706.
When unplugging usb devices such as your debugger and the ESP32 itself, be sure to note the order when re-inserting. Sometimes there's an unexpected recognition delay on the VM; you'll want to be consistent (e.g. having ESP32 on /dev/ttyUSB0). Trust me, you'll wait all day long trying to flash ESP32 firmware onto your JTAG debugger if it somehow ends up as /dev/ttyUSB0
If you see something like this there's a good change the JTAG adapter ended up on the default ttyUSB0 port. Edit the config or change the plug order.
>
If you see something like this there's a good change the JTAG adapter ended up on the default ttyUSB0 port. Edit the config or change the plug order.
>
gojimmypi@ubuntu : ~/esp/hello_world
0 $ sudo chmod 777 /dev/ttyUSB0
[sudo] password for gojimmypi:
gojimmypi@ubuntu : ~/esp/hello_world
0 $ make flash
Flashing binaries to serial port /dev/ttyUSB0 (app at offset 0x10000)...
esptool.py v2.0-beta3
Connecting........_____....._____....._____....._____....._____....._____....._____....._____....._____....._____
A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header
/home/gojimmypi/esp/esp-idf/components/esptool_py/Makefile.projbuild:52: recipe for target 'flash' failed
make: *** [flash] Error 2
Debugging the ESP32 can be rather finicky; I spent a ton of time on the thread problem https://github.com/espressif/openocd-esp32/issues/10
If you don't see the debug console in VSCode: see View - Debug Console (Ctryl-Shift-Y)
Somehow over the course of time, my old friend "ifconfig" fell out of vogue, It is now simply "ip a". I don't really like it; harder to read. So you may need to install net-tools if ifconfig does nto work for you to determine ip addy.
Don't leave OpenOCD / GDB running when recompiling.
Don't leave OpenOCD running when disconnecting JTAG or ESP32. When the process gets stuck, closing the putty window does NOT terminate the process. Regular kill does not work. You'll need -SIGKILL
gojimmypi@ubuntu : ~/esp
1 $ ps aux | grep openocd
root 25099 0.0 0.3 54796 3964 pts/8 S+ 10:30 0:00 sudo ../src/openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f target/esp32-single-thread.cfg
root 25100 0.5 0.4 28660 4252 pts/8 Sl+ 10:30 0:16 ../src/openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f target/esp32-single-thread.cfg
gojimmy+ 25176 0.0 0.0 14224 936 pts/9 S+ 11:23 0:00 grep --color=auto openocd
gojimmypi@ubuntu : ~/esp
0 $ sudo kill -SIGKILL 25100
[sudo] password for gojimmypi:
gojimmypi@ubuntu : ~/esp
0 $ ps aux | grep openocd
gojimmy+ 25180 0.0 0.0 14224 984 pts/9 S+ 11:24 0:00 grep --color=auto openocd
gojimmypi@ubuntu : ~/esp
Resources, Inspiration, and Other Links:
- github Remote Debugging of ESP32 causes EXCEPTION: System.ArgumentOutOfRangeException #763 (many thanks to pieandcakes for help!)
- https://www.cyberciti.biz/faq/kill-process-in-linux-or-terminate-a-process-in-unix-or-linux-systems/
- https://gojimmypi.blogspot.com/2017/05/vscode-remote-debugging-of-embedded.html
- https://github.com/espressif/esp-idf/issues/303
- https://github.com/FHFS/esp-idf-VSCode-template
- Olimex Support Forum » ARM » JTAG » Olimex ARM-USB-OCD-H with OpenOCD - libusb_open() failed with LIBUSB_ERROR_NOT_S
- stackoverflow - OpenOCD debugging multiple devices at once
- allaboutcircuits - Getting Started with OPENOCD Using FT2232H Adapter for SWD Debugging
- Issues with slashes in VSCode config files
- yolinux.com GDB Commands
- openocd.org General Commands
- stackify.com Top Visual Studio Code Extensions: 50 Powerful Tools to Make Visual Studio Even More Useful
- Microsoft vscode tips and-tricks
- FHFS esp-idf-VSCode-template
- VSCode Integrated Terminal
- Building toolchain on Raspberry Pi fails
- Using Raspberry Pi as ESP32 development platform
Completely unrelated, but cool:
