Sunday, August 6, 2017

OpenWrt Remote Network WireShark Packet Feed

After my last post on installing OpenWrt on a Cisco/LinkSys EA3500 to stream RTL-SDR data, I was inspired to investigate other "interesting" data to stream from a WiFi hub. In particular I was hoping to sniff some ESP8266/ESP32 packets during development.

 I had an old Linksys E1200 on hand. Things looked a little bleak as the OpenWrt site seemed to indicate there were some issues. However the router noted there was apparently slightly different than mine (I have the Cisco logo) so I thought I'd give it a try.

At the bottom of the Linksys TechData page I found a link to the OpenWrt firmware: openwrt-15.05-brcm47xx-mips74k-linksys-e1200-v2-squashfs.bin and downloaded it (there are other downloads here in case you don't have a V2, etc). Install was easy from the stock Linksys firmware. I was able to easily telnet to IP 192.168.1.1 - a fixed address when plugging the router into my test network, not via DHCP. This also meant it had no clue as to DNS, default router, etc. Surprisingly LuCI came pre-installed with the bin image. I still edited the config file manually.

You'll probably need to use vi to edit the config files if you don't use LuCI:

i          - to insert
[esc] :qw  - to same and exit
I added only 2 lines to the /etc/config/network file - the gateway and dns settings:
config interface 'lan'
        option ifname 'eth0.1'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option gateway '192.168.1.10'
        option dns '8.8.8.8'
        option ip6assign '60'
Reload the network settings:

/etc/init.d/network reload
Try to ping google.com (or anything on the internet) from the router. If successful, install a few things:
opkg update
opkg install nano # install nano if you don't know VI/VIM very well
opkg install tcpdump # this is the important part that will be called remotely from you client wireshark

On your local Ubuntu:
sudo apt-get install wireshark
sudo mkfifo /tmp/pipe
sudo chmod 777 /tmp/pipe

Or on Windows WSL (Ubuntu on Windows 10):
sudo apt-get install wireshark
sudo apt-get install x11-apps
export DISPLAY=:0 # you may wish you put in this your ~/.bashrc
sudo pkexec env DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR wireshark
Then for wireshark:
wireshark -k -i /tmp/pipe & ssh root@192.168.1.1 "tcpdump -i any -s 0 -U -w - not port 22" > /tmp/pipe
It should look something like this. There's initially a failure message, but then a few moments later a password prompt. One you login, all the WiFi router packets should be forwarded to the your local network packet capture!
0 $ wireshark -k -i /tmp/pipe & ssh root@192.168.1.1 "tcpdump -i any -s 0 -U -w - not port 22" > /tmp/pipe
[3] 4709
Failed to connect to generic netlink.
root@192.168.1.1's password:
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes

For Windows 10 WSL: I downloaded XMing X-windows client. Be careful,there are a lot of fake download links there. If you know of a better place to download XMing or some other Windows X-windows client, please let me know. Despite being 8 years old - XMing seems to work well on Windows 10:


Pinging google from local router


viewing local router traffic on remote WSL WireShark w/XMing






Saturday, July 22, 2017

OpenWRT on EA3500 with RTL-SDR Stream

Recently I saw an interesting rtl-sdr article with a YouTube video on setting up a used Cicso LinkSys EA3500 router with OpenWrt and the RTL-SDR, (the second part of the video is here).

Cisco EA3500 with NooElec RTL-SDR
The video was so interesting, that I found a used EA3500 for sale on ebay and immediately purchased it to try this myself! (not a single one of my routers has a USB interface. Go figure) There are also used ones on amazon as well.

Although it was the topic of SDR caught my attention, it is still a pretty good video on OpenWrt.

The cool thing is that when done - I can move my RTL-SDR device anywhere within range of WiFi! Sweet. No longer limited to USB cable distance of my computer.

For stock firmware, there's no username; default password is admin

You should note that when the router first boots with stock firmware and factory reset, there's NO security on WiFi.

I did this on Windows 10, with the Ubuntu WSL for ssh access and other unix tools like iperf.

My wired test network is setup specifically to allow new devices with the default 192.168.1.1 address to work without having to unplug and re-plug my computer for many devices. However for this exercise I need to connect to OpenWrt before it is configured. So my setup is a little different than the video:


[internet] -- [switch: 192.168.1.x ] --- [yellow ES3500 Internet port]
                                         [blue ES3500 Ethernet port  ] --- [my computer]


While videos are cool, they make for poor reference material for later use. It is difficult to "scroll" though a video looking for a particular technical item. I'll keep track of all the details here.

The first reference is to the WikiDevi Linksys EA3500 page, with a link to the OpenWrt page for the Linksys EA3500. Despite the name factory.bin - this is NOT the stock Linksys software. This is the file to download for OpenWrt. There's more information on the Techdata: Linksys3500 page. That's where you can find a link to Firmware OpenWrt Install URL.

The first thing I did was brick my router. :(  I loaded that openwrt-kirkwood-linksys-audi-squashfs-factory.bin mentioned above. It did not go well. Fortunately there's a magic sequence of events to restore it. Thanks mikemccartney for posting these instructions that worked for me:


1 Plug in the EA3500/4500
2 Power led will blink rapidly
3 Power led will turn off
4 As soon as the power led turns off, unplug the router
5 Wait a few seconds
6 Plug the router back in and repeat the above process

Do that three times, on the fourth time keep the router plugged in and let it boot, hopefully you will now have a functioning router again.

I'm just glad I didn't have to figure out how to JTAG the firmware onto the router.

This time when loading openwrt-kirkwood-linksys-audi-squashfs-factory.bin things went much better.

So back to the video instructions: there's no default web UI when OpenWrt first boots. To initially access the new firmware, use: ssh root@192.168.1.1 and it should look something like this:


0 $ ssh root@192.168.1.1
The authenticity of host '192.168.1.1 (192.168.1.1)' can't be established.
RSA key fingerprint is 6e:6d:a4:4e:83:9e:54:10:b3:be:c0:76:bf:eb:aa:e3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.1' (RSA) to the list of known hosts.


BusyBox v1.24.2 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 DESIGNATED DRIVER (Bleeding Edge, 50107)
 -----------------------------------------------------
  * 2 oz. Orange Juice         Combine all juices in a
  * 2 oz. Pineapple Juice      tall glass filled with
  * 2 oz. Grapefruit Juice     ice, stir well.
  * 2 oz. Cranberry Juice
 -----------------------------------------------------
root@OpenWrt:~#
Next set password with
passwd
command:
root@OpenWrt:~# passwd
Changing password for root
New password:
Retype password:
Password for root changed by root

If you are familar with VI/VIM you can skip this step, otherwise you can install nano:
opkg update

You may wish to save you original configs:
cp  /etc/config/network  /etc/config/network.bak
cp  /etc/config/wireless /etc/config/wireless.bak
Edit /etc/config/network that originally looks like this:
config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdbc:cc57:906d::/48'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6'

He edits the network from 192.168.1.1 to 192.168.2.1
config interface 'lan'
        option ipaddr '192.168.2.1'
and changes the wan interface from eth1 to wlan0 for both.
config interface 'wan'
        option ifname 'wlan0'

config interface 'wan6'
        option ifname 'wlan0'

Then edit /etc/config/wireless that for a default config, that originally looks like this.
config wifi-device  radio0
        option type     mac80211
        option channel  11
        option hwmode   11g
        option path     'mbus/mbus:pcie-controller/pci0000:00/0000:00:01.0/0000:01:00.0'
        option htmode   HT20
        # REMOVE THIS LINE TO ENABLE WIFI:
        option disabled 1

config wifi-iface
        option device   radio0
        option network  lan
        option mode     ap
        option ssid     OpenWrt
        option encryption none

config wifi-device  radio1
        option type     mac80211
        option channel  36
        option hwmode   11a
        option path     'mbus/mbus:pcie-controller/pci0000:00/0000:00:02.0/0000:02:00.0'
        option htmode   HT20
        # REMOVE THIS LINE TO ENABLE WIFI:
        option disabled 1

config wifi-iface
        option device   radio1
        option network  lan
        option mode     ap
        option ssid     OpenWrt
        option encryption none

Remove the channel selection and enable wifi (by setting disabled to false):
config wifi-device  radio0
        # option channel  11
        option disabled 0
And edit the wifi-iface section for 2GHz:
config wifi-iface
        option device   radio0
        option network  wan
        option mode     sta
        option ssid     yourssid
        option encryption psk2
        option key      yourwifipassword
Edit the wifi-iface section for 5GHz in a similar manner if you have one (I do not). The reboot, wait, and ssh to the NEW IP address:
reboot;exit
# wait...
ssh root@192.168.2.1
Once everything is configured, the cable between the yellow Ethernet port and the switch can be removed, as the router is now a station

[internet] -- [WiFi (same as above switched network)]   
                            [WiFi]  ---- [blue ES3500 Ethernet port  ] --- [my computer]
Other things installed in the video include installing LuCI (the graphical web gui). Note the video installed regular luci; I prefer the luci-ssl, even though the browser will complain that it is not a secure connection (no trusted root for self-signed cert)


opkg update
# opkg install luci # the non-ssl version
opkg install luci-ssl
/etc/init.d/uhttpd start
/etc/init.d/uhttpd enable
Then see which RTL packages are available:
opkg list | grep rtl
He then goes on to install rtl-sdr
opkg install rtl-sdr
which rtl_fm
which rtl_tcp
rtl_tcp -h
Plugging in my NooElec RTL-SDR that I received for Christmas (what a cool gift!!) I see this latest entry with dmesg:
[ 2819.934581] usb 1-1: new high-speed USB device number 2 using orion-ehci
Then simply run rtl_tcp:
root@OpenWrt:~# rtl_tcp
Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Tuned to 100000000 Hz.
listening...
Use the device argument 'rtl_tcp=127.0.0.1:1234' in OsmoSDR (gr-osmosdr) source
to receive samples in GRC and control rtl_tcp parameters (frequency, gain, ...).
Other things installs in the video:

 
opkg list | grep dump
opkg install dump1090
opkg install iperf
Part two of the video starts out a bit confusing... but skip the first 7 minutes and pick up at the OpenWrt login prompt. There's a reminder on restarting WiFi:

wifi down; wifi up
# or
ifconfig wlan0 down
ifconfig wlan0 up

Basically to ensure there's an IP address listed for wlan0. (the client DHCP address for your WiFi network; remember this device was configured as STA, not AP (above).
As such, the ES3500 is connected to the "internet" via WiFi instead of via the yellow RJ-45 connector. Thus for wireless access, port forwarding is needed. At first, I setup a simple traffic rule that worked:


However I wanted to follow along and use port forwarding instead, like this:

A port forward that looks like it should work, but does not.

Unfortunately, the video already had many of the port forwards already configured. I had a difficult time with getting ssh working. I kept getting these odd connection refused errors:
gojimmypi@ElectronicsDesk : ~
255 $ ssh root@192.168.1.131
ssh: connect to host 192.168.1.131 port 22: Connection refused
Now "connection refused" is an interesting error. It says things are working but being rejected. I was sure I had it configured correctly. As it turns out there's a feature specifically to allow ssh under System - Administration:


Even setting the "allow remote hosts", I continued to get the port refusal error. So in the video, he connects to the client STA address of the EA3500. Yet when on the local 192.168.2.x network, I was ssh'ing into the router IP address: 192.168.2.1 and not its client STA address of 192.168.1.131 - so I used that as a port forward instead. Note that it shows up in the drop-down as "(Openwrt,lan)" next to the IP addy


And so once that is working, several other ports also need to be forwarded:

Note that when moving your client computer between the ES3500 network (192.168.2.x) and the local wired/wireless network (192.168.1.x) - there's a delay of up to several minutes in Windows obtaining the new address and getting everything setup, during which you'll see messages like this when attempting ssh:


ssh: connect to host 192.168.1.131 port 22: Software caused connection abort
ssh: connect to host 192.168.1.131 port 22: Resource temporarily unavailable
ssh: connect to host 192.168.1.131 port 22: Network is unreachable

If you still have problems connecting, recheck settings and try reboot. But be patient, it does take some time. Again, note that I believe the port forwards in the video are completely wrong - and the only reason he was able to get it working was the traffic rules. I have no traffic rules - just the port forwards, but to a different address. But then again, I spent the better part of the afternoon playing with this - and the video author sped though the entire process in an impressive 37 minutes apparently with no editing / interruptions. Impressive.
Once that is working, I setup iperf as a server on the EA3500:
iperf -s
In a separate shell, staying local the PC (I also needed to install iperf):
sudo apt-get install iperf
iperf -c 192.168.1.131 -i 3 -t 30
and saw mediocre performance from this "gigabit" router:

------------------------------------------------------------
Client connecting to 192.168.1.131, TCP port 5001
TCP window size:  512 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.143 port 5607 connected with 192.168.1.131 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 3.0 sec  6.75 MBytes  18.9 Mbits/sec
[  3]  3.0- 6.0 sec  8.00 MBytes  22.4 Mbits/sec
[  3]  6.0- 9.0 sec  7.12 MBytes  19.9 Mbits/sec
[  3]  9.0-12.0 sec  7.75 MBytes  21.7 Mbits/sec
[  3] 12.0-15.0 sec  7.38 MBytes  20.6 Mbits/sec
[  3] 15.0-18.0 sec  7.25 MBytes  20.3 Mbits/sec
[  3] 18.0-21.0 sec  7.88 MBytes  22.0 Mbits/sec
[  3] 21.0-24.0 sec  7.62 MBytes  21.3 Mbits/sec
[  3] 24.0-27.0 sec  7.75 MBytes  21.7 Mbits/sec
[  3] 27.0-30.0 sec  7.88 MBytes  22.0 Mbits/sec
[  3]  0.0-30.1 sec  75.5 MBytes  21.1 Mbits/sec
Note that Norton Antivirus popped up and warned about the iperf traffic; the default recommended action was to block it. Your AV may simply quietly block it. So if ssh and the web interface are working but not iperf; check your antivirus software. YMMV.

The video moves on to trying performance with a 5GHz band. I don't have any other 5GHz routers, so I could not test this. Important thing to note is that he ended up apparently needed to power cycle the router when changing from 2.5GHx to 5GHz.

So on to SDR "Sharp" (which was also already install in the video). You can download the AirSpy SDR# here. Norton AV gave me a ton of grief on many of the SDR# executables. Here's the config in SDR#. The only thing is setting the IP address for RTL-SDR (tcp):


Although the original pic above has the RTL-SDR plugged directly into the router, it is probably a good idea to use an extension cord, as shown in the video to get the receiver a bit farther away.

Note that we are port forwarding the br-lan address: 192.168.2.1 from the 192.168.1.x network. Thus we need to tell rtl_tcp to listen on that address:

rtl_tcp -a 192.168.2.1 -f 95300000
Press play button in SDR# and voila!



Resources, Inspiration, Credits, and Other Links:





Sunday, July 16, 2017

Updating adafruit.io ESP8266 MQTT code

While working on my flow and pressure project, I stumbled upon a notice that Adafruit planned to change the SSL info for adafruit.io. I had been using this (beta) site for mqtt telemetry. And so today... sure enough: my code is broken & no longer connecting.


I was hoping it was a simple matter of updating the SSL fingerprint to a new value.

const char* fingerprint = "26 96 1C 2A 51 07 FD 15 80 96 93 AE F7 32 CE B9 0D 01 55 C4";
No luck there, as the fingerprint did not change - but the declarative name did. <sigh> Well, so the github repository is here. But I have been blissfully ignorant of exactly where everything is stored. A mixed blessing in allowing me focusing on my code, but sometimes scary in not knowing how everything works. Alas no-one can know everything. So back to the ESP8266 crutch: The Arduino IDE! Click Sketch - Include Library - Manage Libraries. (it is the top-most menu item). Find the installed adafruit mqtt library and click update!


If like me - you are using VisualMicro, I believe it is usually best to exit and restart Visual Studio.


It didn't work. Not only does the connection still fail, but the needed code was apparently not actually updated (the drawback of hiding libraries in a GUI). I know it is not working as the new fingerprint is defined with AIO_SSL_FINGERPRINT and the old one with fingerprint, and there's no compile error without the AIO_SSL_FINGERPRINT. So ya, there appears to be a new include file: adafruitio_definition. I have no such file on my entire system. Well, seems that the Arduino IDE was not successful in getting the really latest adafruit mqtt library.
So the issue here is more than the MQTT library... I also needed to update the Adafruit Arduino IO (listed immediately above in the screen snip). Again exit / restart Visual Studio.

Next, I saw this error:
AdafruitIO.h: 22:31: fatal error: ArduinoHttpClient.h: No such file or directory
   #include "ArduinoHttpClient.h"
   compilation terminated
Seems that the latest Adafruit MQTT also needs the new (apparently experimental at this time) ArduinoHttpClient library. So I installed that from Arduino IDE as well. And restarted Visual Studio. So ya, just as described here. (once you kow that the Arduino IO library is required by new new MQTT.

Ok, so now what? The github repository sample code for adafruitio secure esp8266 is still from last year. No tweets from adafruitio. No other messages on the forum. Blog apparently has "major outage".


End of story for today on this topic. Hopefully more documentation and examples will be available soon.

Saturday, July 15, 2017

ESP8266 OTA Water Pressure Flow Monitor

I've been working with the ESP8266 for quite some time now, but I've only programmed it while it was sitting right at my desk and having it plugged into a USB port. A variety of projects that I have developed have always been convenient to test. However my latest creation involves water. I could not quite imagine how I'd test this at my workbench, nor did it seem very appealing to be programming in the garage. In the summer. In California.

Today I discovered how amazingly simply it is to do "Over The Air" (OTA) programming of the ESP8266, thanks to this really cool Random Nerd Tutorial - ESP8266 OTA Updates with Arduino IDE | Over the Air.

I've been wanting to hook up these pressure and flow sensors that I found on ebay to monitor water usage. I also needed a few fittings that I found on Amazon. I purchased there as I could return them free, no hassle if there was a problem; I had a dreadful time finding anything like this at the local hardware store. I needed this Stainless Steel 316 Cast Pipe Fitting, Hex Bushing, MSS SP-114, 3/4" X 1/4" NPT Male X Female adapter for the pressure sensor, and thisStainless Steel 316 Cast Pipe Fitting, Tee, Class 150, 3/4" NPT Female so that I could hook up both the pressure sensor and flow meter at the same time. The result is a male-female connector that I can slip inline.


You may wonder why I would be interested in something as boring as water pressure. Well, not long ago I discovered my pressure regulator quietly failed. Let's just say I will never again ignore water pressure! There's a reason for the regulator - and the maximum ratings on the water heater. The water softener. The reverse osmosis system.... For only about 10 bucks, you can keep an eye on pressure the  manual, old-school way with this Rain Bird P2A Water Pressure Test Gauge - but what fun is that when there's an electronic gizmo option?

I admit I had some temporary disappointment at the beginning of Step 3 of the OTA tutorial, where I expected to see the the esp8266-xxxxxx at my_esp_ip_address port. I did not. I was not able to see this until exiting and restarting Arduino. No worries, onward!

The next step is to program in some other environment other than the Arduino IDE. I am quite addicted to Visual Studio. I have both the VisualGDB and VisualMicro add-ins. I like VisualMicro in that I can take Arduino code samples and immediately start coding. I have yet to figure out how to do this with VisualGDB. The last time I asked about it, there was no hope on the horizon for Arduino support in VisualGDB,

Initially, it appears there's no option other than the Arduino IDE to actually program OTA. I could find nothing in Visual Studio where it could "see" the pseudo-serial-over-ethernet port. Ha! But fortunately there's a verbose output from the Arduino IDE. In particular this last command shown before uploading:

python.exe C:\Users\gojimmypi\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0/tools/espota.py -i 192.168.1.103 -p 8266 --auth= -f C:\Users\GOJIMM~1\AppData\Local\Temp\arduino_build_690448/sketch_jul15a.ino.bin

The important thing here is that there's a python script called espota.py that sends a binary file to the OTA client. (side note: simply genius implementation in less than 9K!) The paths are crazy long, but the command is essentially this:

python.exe espota.py -i 192.168.1.101 -p 8266 --auth= -f yourfile.bin
Pasting the same OTA sample code into a VisualMicro Ardino Project in Visual Studio... then compiling, shows the output path of the binary:
"C:\Users\gojimmypi\AppData\Local\arduino15\packages\esp8266\tools\esptool\0.4.9/esptool.exe" -eo "C:\Users\gojimmypi\AppData\Local\arduino15\packages\esp8266\hardware\esp8266\2.3.0/bootloaders/eboot/eboot.elf" -bo "C:\Users\gojimmypi\AppData\Local\Temp\VMBuilds\WaterPressureFlow\esp8266_generic\Release/WaterPressureFlow.ino.bin" -bm qio -bf 40 -bz 4M -bs .text -bp 4096 -ec -eo "C:\Users\gojimmypi\AppData\Local\Temp\VMBuilds\WaterPressureFlow\esp8266_generic\Release/WaterPressureFlow.ino.elf" -bs .irom0.text -bs .text -bs .data -bs .rodata -bc -ec
This one is an even longer and crazier command. Really the only thing we are interested in is where the output file is stored. In my case it is:
"C:\Users\gojimmypi\AppData\Local\Temp\VMBuilds\WaterPressureFlow\esp8266_generic\Release/WaterPressureFlow.ino.bin"
There's some sloppy use of mixing forward and back-slashes; for Windows I highly recommend editing them and making them all backslashes.

So now using the Arduino OTA command with the destination of the VisualStudio binary, we can develop OTA with Visual Studio! My command looks like this:
python.exe C:\Users\gojimmypi\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\tools\espota.py -i 192.168.1.103 -p 8266 --auth= -f C:\Users\gojimmypi\AppData\Local\Temp\VMBuilds\WaterPressureFlow\esp8266_generic\Release\WaterPressureFlow.ino.bin
I thought perhaps this could be fully automated by adding the command to the Post Build Event in VisualStudio:
 
However I was unable to get the command to trigger. This article on Specifying Build Events at Microsoft seems to indicate that last option "In Excluded From Build, specify Yes if you do not want the event to run". However my prompt is "Use in Build". I tried both Yes and No - and the OTA command appears to never have been attempted. Perhaps this is because it is a VisualMicro project. No worries, still just as easy to use the up-arrow in a DOS box to re-upload freshly compiled code.

I'm still hopeful that I will be able to do this in a single build operation. The VisualMicro site has a page indicating that build events are now supported. I've submitted a question to their support forum to find out. It appears the only option is a "quite convoluted and difficult to test" option. Hopefully they have the built-in events working.
And indeed YES! Just minutes after posting my question on the forum, I received a response! A simple matter of right-click on project - add - Add Local Board.txt with this text

# WaterPressureFlow build property overrides
#
# local project board.txt overrides
network.port=8266
as described here. So no need for a convoluted solution or manual specification of build event. It just works! Be sure to press "enter" when entering an IP address for serial port (otherwise it won't stick). Now the VisualMicro "Build & Upload" sends OTA data. Sweet. :)



Tuesday, July 4, 2017

Converting a Blue Pill STM32F103 board to a Black Magic Probe

Recently, I learned about the really awesome Black Magic Probe - an interesting JTAG and SWD adapter that essentially has its own, built-in OpenOCD server... so you can use only the GDB client to connect directly to this device for debugging! I've been meaning to place an order for one, but then I ran across this article that converts an STM32F103 "Blue Pill" board to a Black Magic Probe!

Edit: I also tried to use the Maple Mini clone, like this one unsuccessfully; Although I was able to flash the bootloader, my computer did not recognized the device when plugged into the mini-USB connector. (apparently I'm not the only one that was unsuccessful with this).

The instructions there appear very straightforward, however as soon as I saw the some of the text, I realized the guy was using a slightly different distro of linux. So I'll add my notes on using Ubuntu. My Ubuntu is in a VMWare Workstation Virtual Machine running on Windows 10. If you have a similar setup - just don't forget to connect the device as needed from host in VM - Removable Devices. The WSL Ubuntu on Windows that I have still does not yet recognize all USB devices.

Another significant issue for me turned out to the fact that the STM32F103C8T6 is (debatable: see my comment here and noted by someone else here) only a 64K device (8K for bootloader leaving 56K for app)... and the Black Magic Probe now compiles to 57K. If it was easy, it would be no fun, eh?

To get started, ensure the tool chain is installed:


sudo apt-get update
sudo apt-get install git --assume-yes
sudo apt-get install gcc-arm-none-eabi --assume-yes
sudo apt-get install dfu-util --assume-yes

cd ~/
mkdir -p workspace
cd ~/workspace/
git clone https://github.com/jsnyder/stm32loader.git # download stm32loader
git clone --recursive https://github.com/blacksphere/blackmagic.git # download Black Magic source
#git clone --recursive https://github.com/rogerclarkmelbourne/Arduino_STM32.git # optional, has stm32flash
# optional stm32flash (64 bit version)
mkdir -p tools
cd tools
wget https://github.com/rogerclarkmelbourne/Arduino_STM32/raw/master/tools/linux64/stm32flash/stm32flash
chmod +x stm32flash
cd ~/workspace/blackmagic
make
cd src
make clean && make PROBE_HOST=stlink


when all looks good, see if you can talk to your board:

sudo apt-get install python-serial
sudo chmod 777  /dev/ttyUSB0
cd ~/workspace/blackmagic/src/
python ~/workspace/stm32loader/stm32loader.py -p /dev/ttyUSB0

it should return a result like this:
Bootloader version 22
Chip id: 0x410 (STM32 Medium-density)

Ready to flash! However - note that this is where my instructions are different.

cd ~/workspace/blackmagic/src/
python ~/workspace/stm32loader/stm32loader.py -p /dev/ttyUSB0 -e -w -v blackmagic_dfu.bin

The first time I tried, there was an error (also not the jumper setting needed for BOOT0=1:
Can't init. Ensure that BOOT0 is enabled and reset device
Traceback (most recent call last):
  File "/home/gojimmypi/workspace/stm32loader/stm32loader.py", line 436, in 
    bootversion = cmd.cmdGet()
  File "/home/gojimmypi/workspace/stm32loader/stm32loader.py", line 118, in cmdGet
    if self.cmdGeneric(0x00):
  File "/home/gojimmypi/workspace/stm32loader/stm32loader.py", line 115, in cmdGeneric
    return self._wait_for_ask(hex(cmd))
  File "/home/gojimmypi/workspace/stm32loader/stm32loader.py", line 88, in _wait_for_ask
    raise CmdException("NACK "+info)
__main__.CmdException: NACK 0x0
So just as the error hint said, I pressed the reset button and tried again. Success! Output looks like this:
Bootloader version 22
Chip id: 0x410 (STM32 Medium-density)
Write 256 bytes at 0x8000000
Write 256 bytes at 0x8000100
  [..snip..]
Write 256 bytes at 0x8001900
Write 256 bytes at 0x8001A00
Read 256 bytes at 0x8000000
Read 256 bytes at 0x8000100
  [..snip..]
Read 256 bytes at 0x8001900
Read 256 bytes at 0x8001A00
Verification OK
DFU loader done! Be sure to move jumpers back to default of "0" and press reset again. The instructions that I followed indicated that the USB device should be disconnected and reconnected. Not much interesting or different there upon re-insertion. Device is still a cp2104 on /dev/ttyUSB0
[927124.098974] usb 2-2.1: new full-speed USB device number 18 using uhci_hcd
[927124.418782] usb 2-2.1: New USB device found, idVendor=10c4, idProduct=ea60
[927124.418783] usb 2-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[927124.418784] usb 2-2.1: Product: CP2104 USB to UART Bridge Controller
[927124.418785] usb 2-2.1: Manufacturer: Silicon Labs
[927124.418786] usb 2-2.1: SerialNumber: 012345678
[927124.424936] cp210x 2-2.1:1.0: cp210x converter detected
[927124.433135] usb 2-2.1: cp210x converter now attached to ttyUSB0 
The first time I ran dfu-util:
dfu-util -d 1d50:6018,:6017 -s 0x08002000:leave -D blackmagic.bin
I received an error:
dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
dfu-util: No DFU capable USB device available
I had 2 new devices: /dev/serial and /dev/ttyUSB0 when plugged in.

The reason for the error? I missed the MOST IMPORTANT PART:

Disconnect everything and use the USB to connect.

The Black Magic probe uses the micro USB connector, NOT the Serial TTL! (oops) Sure enough, the device is now listed in Device Manager:


A word of caution here: if you happen to use different sources of power, beware of ground loops. I highly recommend only 1 source of power (either the USB TTY/UART or onboard Micro USB. This could be particularly important if they are on different physical computers, using a USB hub, etc.

When you remove the TTY/UART and connect the USB port on the Blue Pill, you should see something like this with dmesg:

[937218.226791] usb 2-2.1: USB disconnect, device number 22
[937218.227053] cp210x ttyUSB0: cp210x converter now disconnected from ttyUSB0
[937218.227064] cp210x 2-2.1:1.0: device disconnected
[937223.643635] usb 2-2.1: new full-speed USB device number 23 using uhci_hcd
[937223.959267] usb 2-2.1: New USB device found, idVendor=10c4, idProduct=ea60
[937223.959269] usb 2-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[937223.959271] usb 2-2.1: Product: CP2104 USB to UART Bridge Controller
[937223.959272] usb 2-2.1: Manufacturer: Silicon Labs
[937223.959273] usb 2-2.1: SerialNumber: 01234567
[937223.967154] cp210x 2-2.1:1.0: cp210x converter detected
[937223.973711] usb 2-2.1: cp210x converter now attached to ttyUSB0
[937258.140246] usb 2-2.1: USB disconnect, device number 23
[937258.144289] cp210x ttyUSB0: cp210x converter now disconnected from ttyUSB0
[937258.144327] cp210x 2-2.1:1.0: device disconnected

[938012.539892] usb 2-2.1: new full-speed USB device number 24 using uhci_hcd
[938012.886453] usb 2-2.1: New USB device found, idVendor=1d50, idProduct=6017
[938012.886455] usb 2-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[938012.886472] usb 2-2.1: Product: Black Magic (Upgrade) for STLink/Discovery, (Firmware v1.6.1-43-g984f8b3)
[938012.886474] usb 2-2.1: Manufacturer: Black Sphere Technologies
[938012.886474] usb 2-2.1: SerialNumber: 76543210
Unfortunately dfu-util -l is not happy:

dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

dfu-util: Cannot open DFU device 1d50:6017
To confirm the device is there,lsusb shows the device as "OpenMoko":

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 025: ID 1d50:6017 OpenMoko, Inc.
Bus 002 Device 003: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
Bus 002 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
You can also view just the device you are looking for with the -d parameter:
lsusb -d 1d50:
(don't forget the trailing colon ":"!

Ok, not obvious - but the problem here is one again permissions. So run the command with sudo:

sudo dfu-util -l
If successful, the output should look like this:

dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

Found DFU: [1d50:6017] ver=0100, devnum=27, cfg=1, intf=0, alt=0, name="@Internal Flash /0x08000000/8*001Ka,056*001Kg", serial="76543210"




Once you can see the device, run dfu-util with sudo:


sudo dfu-util -d 1d50:6018,:6017 -s 0x08002000:leave -D blackmagic.bin
I received an error, that sure looks like an out of memory issue:

dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1d50:6017
Run-time device DFU version 011a
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 1024
DfuSe interface name: "Internal Flash   "
Downloading to address = 0x08002000, size = 58364
dfu-util: Last page at 0x080103fb is not writeable
So I found other instructions relating to STM Discovery as a Black Magic Probe that indicated dfu_upgrade.bin needed to be loaded (I was willing to try anything!)

sudo dfu-util -d 1d50:6018,:6017 -s 0x08002000:leave -D dfu_upgrade.bin
The results are a bit more promising:

dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1d50:6017
Run-time device DFU version 011a
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 1024
DfuSe interface name: "Internal Flash   "
Downloading to address = 0x08002000, size = 6764
Download        [=========================] 100%         6764 bytes
Download done.
File downloaded successfully
Transitioning to dfuMANIFEST state
But even with that - I was completely unable to use dfu-util to program the Blue Pill with a 57K blackmagic.bin file. Even after power cycle, reset, etc. So I put the board back into USART programming mode (BOOT0 jumper=1, BOOT1 jumper=0) and used the ST Flash Loader app on Windows. The device is initially detected as a 65K device:


However on the next screen, apparently you can force it into 128K mode:


The select the blackmagic.bin file, click the "Erase necessary pages", select an address of 0x8002000 and check the "Verify after download" checkbox:


If successful, you should see:

I should point out that several times I received a verify error at this last verify step (I was really convinced I had a 64K-only device). After trying again the next day - things started working immediately and I never again saw the verify error, even after doing a full erase and starting over. YMMV.

Keep in mind that the fact that we are using 128K on a 64K device is rather dangerous. In all likelihood, the manufacturer does an exhaustive memory test: throws away the ones with an error in the first 64K, market the ones with an error in the second 64K as only a 64K STM32f103C8T6 device, and label the ones with no error as 128K STM32f103CBT6 devices. Although this may be a fun Saturday afternoon project, if you are serious about actually using and depending on a BlackMagic probe for real debugging, consider buying one. (besides, always good to support the developer).  When you are elbows-deep in debugging your own project, you really don't want a questionable debugger.

So at this point, I'm not even sure the dfu bootloader is needed, if we are loading the blackmagic.bin file this way.

My particular debugging experiment uses the STM Smart V2 board as a target, and using my STM32 to ST7735 TFT LCD display example with the source on github. This is somewhat of a convoluted setup, as the project is in Visual Studio 2017 (on Windows 10) with the VisualGDB Extension from Sysprogs.

As previously mentioned, I'm running my Ubuntu in a VM, and I have samba installed so that on Windows I can map a drive  (C$ is root  /):

net use z: \\192.168.1.30\c$ /user:gojimmypi

Debugging connections from Blue Pill board configured as a Black Magic Probe (micro usb connected to Ubuntu):

Connecting the Blue Pill Black Magic Probe to an STM Smart V2 board:





cd  ~/workspace
git clone https://github.com/gojimmypi/STM32-ST7735.git

then open the solution Z:\home\gojimmypi\workspace\STM32-ST7735\STM32-ST7735.sln in Visual Studio and compile.


cd  ~/workspace/STM32-ST7735/STM32-ST7735
arm-none-eabi-gdb -d ./ -f ~/workspace/STM32-ST7735/VisualGDB/Debug/STM32-ST7735 -tui
ls /dev/ttyACM* -al

then in GDB, connect with the first ttyACM[n] device (the second one is a serial port). Mine were originally ttyACM0 and ttyACM1, but then later changed to ttyACM1 and ttyACM2 (no idea how/why). I even saw an instance while debugging that ttyACM1 changed to ttyACM0 yes leaving me with (ttyACM0 and ttyACM2).Weird.
Anyhow, here are some useful GDB commands to get started:

target extended-remote /dev/ttyACM1
monitor swdp_scan
attach 1


If successful you should see something like this:

GNU gdb (7.10-1ubuntu3+9) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-linux-gnu --target=arm-none-eabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/gojimmypi/workspace/STM32-ST7735/VisualGDB/Debug/STM32-ST7735...done.
(gdb) target extended-remote /dev/ttyACM1
Remote debugging using /dev/ttyACM1
(gdb) monitor swdp_scan
Target voltage: unknown
Available Targets:
No. Att Driver
 1      STM32F1 medium density
(gdb) attach 1
Attaching to program: /home/gojimmypi/workspace/STM32-ST7735/VisualGDB/Debug/STM32-ST7735, Remote target
0x08001514 in SetSysClockTo72 () at system_stm32f10x.c:993
(gdb)

sample


Resources, Inspiration, Credits, and Other Links:





Friday, June 9, 2017

VSCode JTAG Debugging of ESP32 - Part 3

(see also hardware setuppart 1, and part 2)

Test drive of OpenOCD WIP branch with VSCode. See: https://github.com/espressif/openocd-esp32/wiki/Work-in-progress-ESP32-dual-core-target

If there's already an openocd-esp32 checked out, save it:
cd ~/workspace
mv openocd-esp32 openocd-esp32-master

Fetch the WIP branch
cd ~/workspace
git clone -b feature/esp32_dualcore  --recursive https://github.com/espressif/openocd-esp32.git
cd openocd-esp32
git status
then follow instructions https://github.com/espressif/openocd-esp32
./bootstrap 
./configure 
make
sudo make install

Modified esp32_dc.cfg to esp32_dcx.cfg

cp  /./home/gojimmypi/workspace/openocd-esp32/esp32_dc.cfg /./home/gojimmypi/workspace/openocd-esp32/esp32_dcx.cfg
Mine looks like this:

# Example configuration file to hook up an ESP32 module or board to a JTAG
# adapter. Please modify this file to your local setup.

# Include the configuration for the JTAG adapter.
# By default, DevKit-J is used.
# Later versions of DevKit-J are known as ESP-WROVER-KIT,
# they can work with the same configuration file.
# If you use a different interface, please edit this to include the
# configuration file of yours.
# source [find interface/ftdi/esp32_devkitj_v1.cfg]

# The ESP32 only supports JTAG.
transport select jtag

# The speed of the JTAG interface, in KHz. If you get DSR/DIR errors (and they
# do not relate to OpenOCD trying to read from a memory range without physical
# memory being present there), you can try lowering this.
# On DevKit-J, this can go as high as 20MHz if CPU frequency is 80MHz, or 26MHz
# if CPU frequency is 160MHz or 240MHz.
adapter_khz 1000

#Source the ESP32 configuration file
source [find target/esp32_dc.cfg]

# The TDI pin of ESP32 is also a bootstrap pin that selects the voltage the SPI flash
# chip runs at. When a hard reset happens (e.g. because someone switches the board off
# and on) the ESP32 will use the current TDI value as the bootstrap value because the
# JTAG adapter overrides the pull-up or pull-down resistor that is supposed to do the
# bootstrapping. These lines basically set the idle value of the TDO line to a
# specified value, therefore reducing the chance of a bad bootup due to a bad flash
# voltage greatly.

# Enable this for 1.8V SPI flash
#esp32 flashbootstrap 1.8
# Enable this for 3.3V SPI flash
esp32 flashbootstrap 3.3

Note there's a discussion for openocd-esp32/issues/6 that mentions the flash voltage in config file:
That depends on the module which is used on the dev board. Most WROOM modules use 3.3V flash, while WROVER modules use 1.8V flash.

For a JTAG debugger, I'm using the Olimex (default file, see esp32 config for some settings). I could not get the default 20MHz to work, so lower this setting if there are problems. Sample above is using 1MHz, but I have been able to get 10MHz to work reliably. Actual max speed will depend on JTAG cable type characteristics (length, capacitance, shielding, etc)
#
# Olimex ARM-USB-OCD-H
#
# http://www.olimex.com/dev/arm-usb-ocd.html
#

interface ft2232
ft2232_device_desc "Olimex OpenOCD JTAG ARM-USB-OCD-H"
ft2232_layout olimex-jtag
ft2232_vid_pid 0x15ba 0x002b
Modified launch.json in VSCode (different from prior blog). Note changes as specified here. But I omitted last line with single "c".
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "OpenOCD Debug",
            "type": "cppdbg",
            "request": "launch",
            "miDebuggerPath": "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe",
            "program": "Y:/home/gojimmypi/esp/hello_world/build/hello-world.elf",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                },
                {
                    "text": "file 'Y:/home/gojimmypi/esp/hello_world/build/hello-world.elf'"
                },
                {
                    "text": "target remote 192.168.174.130:3333"
                },
                {
                    "text": "monitor reset halt"
                },
                {
                    "text": "thb app_main"
                },
                {
                    "text": "x $a1=0"
                }
            ],
            "externalConsole": false,
            "cwd": "Y:/home/gojimmypi/esp/hello_world/build/",
            "logging": {
                "trace": true,
                "traceResponse": true,
                "engineLogging": true
            }
        }
    ]
}
then launch OpenOCD
cd ~/workspace/openocd-esp32/tcl
sudo openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg  -f ~/workspace/openocd-esp32/esp32_dcx.cfg
Output looks like this:
Open On-Chip Debugger 0.10.0-dev-gc83a89c (2017-06-09-11:42)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
adapter speed: 1000 kHz
force hard breakpoints
Info : clock speed 1000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.slave tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : Target halted. PRO_CPU: PC=0x4008495D (active)    APP_CPU: PC=0x00000000

Note FreeRTOS change for app: Disable "Stop program on scheduler start..."
cd ~/esp/hello_world
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf
make menuconfig

Rebuild and flash. Note I edited my Hello World to loop and not reset.
make clean
make
make flash
Ensure it is working:

make monitor
Output like this:

MONITOR
--- idf_monitor on /dev/ttyUSB0 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0x00
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0008,len:8
load:0x3fff0010,len:4400
load:0x40078000,len:11092
ho 0 tail 12 room 4
load:0x40080000,len:252
0x40080000: _iram_start at ??:?

entry 0x40080034
0x40080034: _iram_start at ??:?

I (49) boot: ESP-IDF v2.0-rc1-854-gba55461 2nd stage bootloader
I (49) boot: compile time 13:14:19
I (49) boot: Enabling RNG early entropy source...
I (67) boot: SPI Speed      : 40MHz
I (79) boot: SPI Mode       : DIO
I (92) boot: SPI Flash Size : 4MB
I (104) boot: Partition Table:
I (115) boot: ## Label            Usage          Type ST Offset   Length
I (138) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (161) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (185) boot:  2 factory          factory app      00 00 00010000 00100000
I (208) boot: End of partition table
I (221) boot: Disabling RNG early entropy source...
I (238) boot: Loading app partition at offset 00010000
I (753) boot: segment 0: paddr=0x00010018 vaddr=0x00000000 size=0x0ffe8 ( 65512)
I (753) boot: segment 1: paddr=0x00020008 vaddr=0x3f400010 size=0x070ac ( 28844) map
I (769) boot: segment 2: paddr=0x000270bc vaddr=0x3ffb0000 size=0x02b00 ( 11008) load
I (800) boot: segment 3: paddr=0x00029bc4 vaddr=0x40080000 size=0x00400 (  1024) load
0x40080000: _iram_start at ??:?

I (822) boot: segment 4: paddr=0x00029fcc vaddr=0x40080400 size=0x12cdc ( 77020) load
I (885) boot: segment 5: paddr=0x0003ccb0 vaddr=0x400c0000 size=0x00000 (     0) load
I (886) boot: segment 6: paddr=0x0003ccb8 vaddr=0x00000000 size=0x03350 ( 13136)
I (906) boot: segment 7: paddr=0x00040010 vaddr=0x400d0018 size=0x26ae8 (158440) map
0x400d0018: _flash_cache_start at ??:?

I (932) cpu_start: Pro cpu up.
I (943) cpu_start: Starting app cpu, entry point is 0x40080ec0
0x40080ec0: call_start_cpu1 at /home/gojimmypi/esp/esp-idf/components/esp32/./cpu_start.c:174

I (0) cpu_start: App cpu up.
I (976) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (998) heap_alloc_caps: At 3FFAE2A0 len 00001D60 (7 KiB): DRAM
I (1018) heap_alloc_caps: At 3FFB6418 len 00029BE8 (166 KiB): DRAM
I (1039) heap_alloc_caps: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (1061) heap_alloc_caps: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1082) heap_alloc_caps: At 400930DC len 0000CF24 (51 KiB): IRAM
I (1103) cpu_start: Pro cpu start user code
I (1161) cpu_start: Starting scheduler on PRO CPU.
Hello world!
I (201) cpu_start: Starting scheduler on APP CPU.
This is ESP32 chip with 2 CPU cores, WiFi/BT/BLE, silicon revision 0, 4MB external flash
Looping in 10 seconds...
Looping in 9 seconds...
Looping in 8 seconds...
Looping in 7 seconds...
Looping in 6 seconds...


On Windows machine, my Y: drive is mapped to the root of the Ubuntu machine (using samba):

net use y: \\192.168.174.130\c$
Then in Windows Explorer, open VSCode from
Y:\home\gojimmypi\esp\hello_world\build

Press F5 to debug! Tada!





VSCode immediately fails with this error if the last line is included in launch.json as noted for gdbinit: (so don't add it that last "c" command)



Unable to start debugging. Unexpected GDB output from command "-interpreter-exec console "c"".


1: (138) LaunchOptions
1: (143) LaunchOptions    
1: (143) LaunchOptions        -enable-pretty-printing
1: (144) LaunchOptions        file 'Y:/home/gojimmypi/esp/hello_world/build/hello-world.elf'
1: (144) LaunchOptions        target remote 192.168.174.130:3333
1: (144) LaunchOptions        monitor reset halt
1: (144) LaunchOptions        thb app_main
1: (144) LaunchOptions        x $a1=0
1: (144) LaunchOptions        c
1: (144) LaunchOptions    
1: (144) LaunchOptions
1: (176) Starting: "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe" --interpreter=mi
1: (189) DebuggerPid=22492
1: (216) ->=thread-group-added,id="i1"
1: (217) ->~"GNU gdb (crosstool-NG crosstool-ng-1.22.0-61-gab8375a) 7.10\n"
1: (217) ->~"Copyright (C) 2015 Free Software Foundation, Inc.\n"
1: (217) ->~"License GPLv3+: GNU GPL version 3 or later \nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to the extent permitted by law.  Type \"show copying\"\nand \"show warranty\" for details.\n"
1: (217) ->~"This GDB was configured as \"--host=i686-host_pc-mingw32 --target=xtensa-esp32-elf\".\nType \"show configuration\" for configuration details."
1: (217) ->~"\nFor bug reporting instructions, please see:\n"
1: (217) ->~".\n"
1: (217) ->~"Find the GDB manual and other documentation resources online at:\n.\n"
1: (217) ->~"For help, type \"help\".\n"
1: (217) ->~"Type \"apropos word\" to search for commands related to \"word\".\n"
1: (217) ->(gdb)
1: (222) <-1001-gdb-set -="" 1:="" on="" target-async="">1001^done
1: (228) ->(gdb)
1: (229) 1001: elapsed time 6
1: (234) <-1002-enable-pretty-printing -="" 1:="">1002^done
1: (239) ->(gdb)
1: (239) 1002: elapsed time 5
1: (241) <-1003-interpreter-exec -="" 1:="" build="" console="" esp="" file="" gojimmypi="" hello-world.elf="" hello_world="" home="">~"Reading symbols from Y:/home/gojimmypi/esp/hello_world/build/hello-world.elf..."
1: (274) ->~"done.\n"
1: (275) ->1003^done
1: (275) ->(gdb)
1: (275) 1003: elapsed time 34
1: (275) <-1004-interpreter-exec -="" 192.168.174.130:3333="" 1:="" console="" remote="" target="">~"Remote debugging using 192.168.174.130:3333\n"
1: (305) ->=thread-group-started,id="i1",pid="42000"
1: (305) ->=thread-created,id="1",group-id="i1"
1: (307) ->~"0x00000000 in ?? ()\n"
1: (307) ->*stopped,frame={addr="0x00000000",func="??",args=[]},thread-id="1",stopped-threads="all"
1: (309) ->1004^done
1: (309) ->(gdb)
1: (318) 1004: elapsed time 42
1: (319) <-1005-thread-info -="" 1:="" 1="">1005^done,threads=[{id="1",target-id="Remote target",frame={level="0",addr="0x00000000",func="??",args=[]},state="stopped"}]
1: (329) ->(gdb)
1: (330) 1005: elapsed time 11
1: (330) <-1006-interpreter-exec -="" 1:="" console="" e="" halt="" monitor="" reason="" reset="" started="" thread:="" thread="" threadid="" type="">@"JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)\n"
1: (366) ->@"JTAG tap: esp32.slave tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)\n"
1: (490) ->@"Target halted. PRO_CPU: PC=0x400D0E1C (active)    APP_CPU: PC=0x00000000 \n"
1: (500) ->@"esp32: target state: halted\n"
1: (542) ->@"esp32: Debug controller was reset (pwrstat=0x5F, after clear 0x0F).\n"
1: (542) ->@"esp32: Core was reset (pwrstat=0x5F, after clear 0x0F).\n"
1: (676) ->@"Target halted. PRO_CPU: PC=0x5000004B (active)    APP_CPU: PC=0x00000000 \n"
1: (691) ->@"esp32: target state: halted\n"
1: (740) ->@"esp32: Core was reset (pwrstat=0x1F, after clear 0x0F).\n"

Even when things seem to be going well, eventually some errors are encountered. If you try to debug and nothing happens, it may be because there are too many breakpoints. The limit is TWO! But perhaps you don't know / remember you even have breakpoints, but see this in the View - Debug Console

1: (1048) <-1020-interpreter-exec -="" 1:="" console="" info="" sharedlibrary="">~"No shared libraries loaded at this time.\n"
1: (1051) ->1020^done
1: (1051) ->(gdb)
1: (1051) 1020: elapsed time 3
E output: {"category":"console","output":"Loaded 'shared libraries loaded at this time.'. Cannot find or open the symbol file.\r\n","data":null,"type":"output"}
Loaded 'shared libraries loaded at this time.'. Cannot find or open the symbol file.
1: (1061) <-1021-thread-info -="" 1:="">1021^done,threads=[{id="1",target-id="Remote target",frame={level="0",addr="0x40000400",func="??",args=[]},state="stopped"}],current-thread-id="1"
1: (1072) ->(gdb)
1: (1072) 1021: elapsed time 11
1: (1078) <-1022-stack-list-frames -="" 0="" 1000="" 1:="">1022^done,stack=[frame={level="0",addr="0x40000400",func="??"}]
1: (1083) ->(gdb)
1: (1083) 1022: elapsed time 4
1: (1086) <--exec-continue -="" 1:="">^error,msg="Warning:\nCannot insert breakpoint 3.\nCannot access memory at address 0x400f4d55\nCannot insert breakpoint 4.\nCannot access memory at address 0x400f4d75\n"
1: (1104) ->(gdb)
E output: {"category":"stderr","output":"ERROR: Warning:\nCannot insert breakpoint 3.\nCannot access memory at address 0x400f4d55\nCannot insert breakpoint 4.\nCannot access memory at address 0x400f4d75\n\r\n","data":null,"type":"output"}
ERROR: Warning:
Cannot insert breakpoint 3.
Cannot access memory at address 0x400f4d55
Cannot insert breakpoint 4.
Cannot access memory at address 0x400f4d75
To resolve this, click on Debug - Remove All Breakpoints in VSCode.

Thursday, May 25, 2017

VSCode JTAG Debugging of ESP32 - Part 2

This in a continuation from part 1. If you need help connecting the ESP32 to JTAG, see this post.

After setting up the ESP32 toolchain and confirming regular GDB debugging is working as described in my previous post, now it is time to configure VSCode for remote debugging of the ESP32.

I've also added Part 3 - using ESP WIP OpenOCD.

Everything is pretty straightforward here, well except for the details of actually debugging in VSCode. I had quite an ordeal with that, as described in github issue  #763: Remote Debugging of ESP32 causes EXCEPTION: System.ArgumentOutOfRangeException.

In the end, it seems that somehow VSCode was confused on the initial *state* of the ESP32 when debugging is first initiated. The first MI commands would get the ESP32 into some sort of weird state "internal to GDB":

C next: {"threadId":-2}
1: (812785) <-1022-exec-next
1: (812793) ->1022^running
1: (812793) ->*running,thread-id="all"
1: (812793) ->(gdb)
1: (812793) 1022: elapsed time 8
1: (812794) ->(gdb)
 R: {"success":true,"message":null,"request_seq":11,"command":"next","body":null,"running":false,"refs":null,"seq":0,"type":"response"}
1: (812903) ->@"esp32.cpu0: Target halted, pc=0x40080E77\n"
E output: {"category":"stdout","output":"@\"esp32.cpu0: Target halted, pc=0x40080E77\\n\"\n","data":null,"type":"output"}
@"esp32.cpu0: Target halted, pc=0x40080E77\n"
1: (812932) ->~"/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.\nA problem internal to GDB has been detected,\nfurther debugging may prove unreliable.\nQuit this debugging session? "
1: (812932) ->~"(y or n) [answered Y; input not from terminal]\n"
E output: {"category":"stdout","output":"/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.\nA problem internal to GDB has been detected,\nfurther debugging may prove unreliable.\nQuit this debugging session? ","data":null,"type":"output"}
1: (812932) ->&"\nThis is a bug, please report it."
/home/ivan/e/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? 

I opened an espressif issue on giuthub for this one: [TW#12876] VSCode Debugging via MI causes internal GDB error  #645. Fortunately - with the help of @pieandcakes - I was able to determine a (yes, somewhat wonky) sequence of steps to persuade VSCode to debug the ESP32!

I have my openocd running on Ubuntu with samba, and have mapped my windows Z: drive to the root of the ubuntu file system. For that, my smb.conf looks like this:

## edit the file /etc/samba/smb.conf and put in these lines at the end (without single # comment markers!)
##*******************************************************************************************************
[c$]
   comment= root
   path=/
   browseable=Yes
   writeable=Yes
   only guest=no
   create mask=0777
   directory mask=0777
   public=no

## also, make sure the authentication line is not commented out
####### Authentication #######
#   security = user
##*******************************************************************************************************

# try to keep away WannaCry
[global]
min protocol = SMB2

First, browse to the Hello World "Build" directory in Windows File Explorer. Right click and click "Open with Code".  The select Debug - Open Configurations. This should create a file called launch.json - in my case, in:

Z:\home\gojimmypi\esp\hello_world\build\.vscode\

Here's my launch.json for debugging the ESP32:


{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "OpenOCD Debug",
            "type": "cppdbg",
            "request": "launch",
            "miDebuggerPath": "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe",
            "program": "Z:/home/gojimmypi/esp/hello_world/build/hello-world.elf",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                },
                {
                    "text": "file 'Z:/home/gojimmypi/esp/hello_world/build/hello-world.elf'"
                },
                {
                    "text": "target remote 192.168.174.129:3333"
                },
                {
                    "text": "monitor reset init"
                }
            ],
            "externalConsole": false,
            "cwd": "Z:/home/gojimmypi/esp/hello_world/build/",
            "logging": {
                "trace": true,
                "traceResponse": true,
                "engineLogging": true
            }
        }
    ]
}
Note that you need to choose between
"miDebuggerServerAddress": "192.168.174.129:3333"
and
"text": "target remote 192.168.174.129:3333"
- but not both! (see that first question in github issue 763). You will, of course - need to edit the IP address to point to your own OpenOCD server, otherwise you will see this error in VSCode:

Unable to start debugging. Unexpected GDB output from command "-interpreter-exec console "target remote 192.168.174.129:3333""


If you don't have a VM or a Linux box, perhaps consider a Raspberry Pi.  :)


The line

            "logging": {
                "trace": true,
                "traceResponse": true,
                "engineLogging": true

is optional, but tremendously helpful in actually seeing what's going on between VSCode and the ESP32.

I created a few gists that can either be put into script files, or copy/pasted for some basic operations. I run each of these in a separate putty shell connection. First, the compile & flash:

#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf

sudo chmod 777 /dev/ttyUSB0

cd ~/esp/hello_world
make flash

In this same "compile window", it can be helpful to run the serial monitor. I like keeping it in the same window as make/building, so that I am sure to stop the monitor when reflashing.

make monitor
Next, run openocd server in yet another putty connection (when successful, the openocd will not exit, rather it continues running as a server for your VSCode GDB connection as a foreground task):

#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf

cd  ~/workspace/openocd-esp32/tcl
sudo openocd -f interface/ftdi/olimex-arm-usb-ocd-h-1MHz.cfg -c "transport select jtag"  -f target/ESP32-RTOS-none.cfg

A successful launch of openocd will look something like this:

0 $ sudo openocd -f interface/ftdi/olimex-arm-usb-ocd-h-1MHz.cfg -c "transport select jtag"  -f target/ESP32-RTOS-none.cfg
[sudo] password for gojimmypi:
Open On-Chip Debugger 0.10.0-dev-g372bb59 (2017-05-19-10:35)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
adapter speed: 1000 kHz
jtag
force hard breakpoints
Info : clock speed 1000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : esp32.cpu0: Target halted, pc=0x400D1158
then the xtensa-esp32-elf-gdb debug client (the find is of course optional) this one with gdb commands in hello_world,gdb:

#!/bin/bash
export PATH=$PATH:$HOME/esp/xtensa-esp32-elf/bin
export IDF_PATH=~/esp/esp-idf
#find /. -name xtensa-esp32-elf-gdb 2>/dev/null

cd ~/esp/hello_world/build
xtensa-esp32-elf-gdb -tui -x hello_world.gdb hello-world.elf


Alternatively launch the xtensa gdb client simply like this (assume paths properly set as shown above):


xtensa-esp32-elf-gdb  hello-world.elf

A successful launch of gdb will look something like this:

gojimmypi@ubuntu : ~/esp/hello_world/build
0 $ xtensa-esp32-elf-gdb  hello-world.elf
GNU gdb (crosstool-NG crosstool-ng-1.22.0-61-gab8375a) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later &lthttp://gnu.org/licenses/gpl.html&gt
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
          &lthttp://www.gnu.org/software/gdb/bugs/&gt.
Find the GDB manual and other documentation resources online at:
          &lthttp://www.gnu.org/software/gdb/documentation/&gt.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from hello-world.elf...done.
(gdb) 

target remote localhost:3333
Remote debugging using localhost:3333
0x400d1158 in esp_vApplicationIdleHook () at /home/gojimmypi/esp/esp-idf/components/esp32/./freertos_hooks.c:52
52              asm("waiti 0");
(gdb)


We just wanted to confirm OpenOCD was working and that we could connect with linux GDB, so exit/quit.

The Windows 10 executable gdb is needed to debug in VSCode on Windows, so as shown on: http://esp-idf.readthedocs.io/en/latest/get-started/windows-setup.html it can be downloaded from here

(mine took HOURS to download)

When ready to start debugging in Windows, locate your xtensa-esp32-elf-gdb.exe, Mine is located in:

C:\msys32\opt\xtensa-esp32-elf\bin

Note the line in the VSCode launch.json:

"miDebuggerPath": "C:/msys32/opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gdb.exe",


That's the one that calls the Windows GDB client to talk to the Ubuntu OpenOCD server. This gives us the best of both worlds: assimilation! Here we can have all the reliable compile of ESP32 code in linux, yet the ability to edit and debug from Windows!

Pay attention to the forward and back-slashes in the launch.json file. See vscode-cpptools issue #706.

The details of these steps are shown in my github issue (in particular the 8th comment after the question) if you want to see more details, but in summary:

Be sure the OpenOCD server is running as described above. Press F5 to start debugging. You should see "Exception has occurred.Trace/breakpoint trap." It appears we are debugging, but we are not. Target (appears to be) running. VSCode thinks it is paused.

If you try to single step here, all bets are off; xtensa GDB crashes, VSCode debugging stops. Game over.

If instead VSCode immediately stops debugging and you see something in the debug log like:

E output: {"category":"stderr","output":"ERROR: Unable to start debugging. Failed to find thread 1 for break event\r\n","data":null,"type":"output"}
ERROR: Unable to start debugging. Failed to find thread 1 for break event
...then go back to part 1 and read about how I slightly modified my esp32.cfg

Otherwise if all is still going according to plan: type -exec continue at the VSCode > prompt.



Observe debug output. Note that despite -exec continue we are now actually in halt state. One of the lines should say:

(16086) ->@"esp32.cpu0: Target halted, pc=0x400F4AB3\n"

Press the VSCode Play button (F5). Observe error:


ERROR: Cannot execute this command while the selected thread is running.


Only after the above steps are completed, now press the pause button. Observe output. There should be no errors.

Press a code-stepping button, such as Shift-F11.... tada! Single step debugging!



For an even better experience - see FHFS comment on espressif/esp-idf issue 303,
"if esp-idf is in the project folder symbols get found automatically. for windows you can also add the msys2, mingw32 and xtensa-esp32-elf bin folders to the path in tasks.json. make monitor has some trouble starting a console, you can edit the idf_monitor.py to output to stderr(task output window), or make a .cmd file with the python command in it and run that command from tasks.
You can see how I work with it esp-idf-VSCode-template"


I'm still working on polishing these instructions. Hopefully you've found this useful. Feedback is welcome and appreciated. Send me a message on twitter, github pull request  or gmail.


Various tips:


GDB tip: Ctrl-x Ctrl-a in gdb to enter TUI mode.

You can press the EN button on the ESP32; it acts like reset and resolves many problems without having to power cycle (which will likely also need permissions re-applied,etc)

Note that as of May 2017, there's an issue in VSCode regarding the direction and quantity of path slashes as note in gitbub vscode-cpptools issue #706.

When unplugging usb devices such as your debugger and the ESP32 itself, be sure to note the order when re-inserting. Sometimes there's an unexpected recognition delay on the VM; you'll want to be consistent (e.g. having ESP32 on /dev/ttyUSB0). Trust me, you'll wait all day long trying to flash ESP32 firmware onto your JTAG debugger if it somehow ends up as /dev/ttyUSB0

If you see something like this there's a good change the JTAG adapter ended up on the default ttyUSB0 port. Edit the config or change the plug order.
>
gojimmypi@ubuntu : ~/esp/hello_world
0 $ sudo chmod 777 /dev/ttyUSB0
[sudo] password for gojimmypi:
gojimmypi@ubuntu : ~/esp/hello_world
0 $ make flash
Flashing binaries to serial port /dev/ttyUSB0 (app at offset 0x10000)...
esptool.py v2.0-beta3
Connecting........_____....._____....._____....._____....._____....._____....._____....._____....._____....._____

A fatal error occurred: Failed to connect to ESP32: Timed out waiting for packet header
/home/gojimmypi/esp/esp-idf/components/esptool_py/Makefile.projbuild:52: recipe for target 'flash' failed
make: *** [flash] Error 2


Debugging the ESP32 can be rather finicky; I spent a ton of time on the thread problem https://github.com/espressif/openocd-esp32/issues/10

If you don't see the debug console in VSCode: see View - Debug Console (Ctryl-Shift-Y)

Somehow over the course of time, my old friend "ifconfig" fell out of vogue, It is now simply "ip a". I don't really like it; harder to read. So you may need to install net-tools if ifconfig does nto work for you to determine ip addy.

Don't leave OpenOCD / GDB running when recompiling.

Don't leave OpenOCD running when disconnecting JTAG or ESP32. When the process gets stuck, closing the putty window does NOT terminate the process. Regular kill does not work. You'll need -SIGKILL


gojimmypi@ubuntu : ~/esp
1 $ ps aux | grep openocd
root      25099  0.0  0.3  54796  3964 pts/8    S+   10:30   0:00 sudo ../src/openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f target/esp32-single-thread.cfg
root      25100  0.5  0.4  28660  4252 pts/8    Sl+  10:30   0:16 ../src/openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f target/esp32-single-thread.cfg
gojimmy+  25176  0.0  0.0  14224   936 pts/9    S+   11:23   0:00 grep --color=auto openocd
gojimmypi@ubuntu : ~/esp
0 $ sudo kill -SIGKILL 25100
[sudo] password for gojimmypi:
gojimmypi@ubuntu : ~/esp
0 $ ps aux | grep openocd
gojimmy+  25180  0.0  0.0  14224   984 pts/9    S+   11:24   0:00 grep --color=auto openocd
gojimmypi@ubuntu : ~/esp


Resources, Inspiration, and Other Links: 


Completely unrelated, but cool:






OpenWrt Remote Network WireShark Packet Feed

After my last post on installing OpenWrt on a Cisco/LinkSys EA3500 to stream RTL-SDR data , I was inspired to investigate other "intere...