LiteX Soft CPU on the ULX3S - reloading firmware without reflashing the soft CPU

My next post (aka notes to self) on getting a RISC-V compiled app onto the ULX3S using LiteX.

TL;DR; press the reset button!

Quite a diversion from my day job with SQL and C# and web apps is this fascinating world of soft CPU's and embedded devices. Here's what I learned this weekend:

In my prior blogs, I considered getting Circuit Python running on the ULX3S. There are a lot of moving parts, so I got started with implementing the RISC-V soft CPU on the FPGA. Recall we cannot run Circuit Python on the ESP32 because of hardware limitations (missing USB OTG), however the ESP32-S2 will (in theory) work with Circuit Python. Next, I rambled on a bit regarding various features of LiteX that I was exploring in my prior blog, in the end creating a soft_cpu.sh script with everything I learned.

Refresher:  Onboard the ULX3S is an ECP5 FPGA. We first need to create a soft CPU bitstream that is loaded onto the ECP5. LiteX also creates a BIOS that gets loaded over the serial port. Ok, all fine and dandy, but I seem to only be able to load that BIOS once and then the terminal becomes unresponsive. This makes sense, as the processor stays in the while loop after liftoff. The bios does not appears to be an RTOS.

ULX3S shown with FPGA/BIOS/firmware programming USB cable

Supposedly that BIOS should be able to reload firmware apps. Well, there's this mystery boot_helper. What does this extern do? I suspect that's what loads the BIOS. Then what? There's this 4 line boot-helper-picorv32.S file. Not much help there. The "What is LiteX" introduction does seem to indicate that the development cycle includes reloading the soft CPU, as the "design flow in a nutshell" indicates going back to step 1. I'd like to go back to step 6, skipping the re-synthesis of the soft CPU every time I change the firmware.

The LiteX for the ULX3S has a serial port configured as Tx on Pin L4 (FTDI RxD) and Rx on Pin M1 (FTDI TxD). See the schematic. These pins are not available on either of the external ULX3S headers.

FTDI RxD/TxD USB Serial on ULX3S FPGA pins L4 and M1

I had thought perhaps the BIOS would be looking on a pair of pins implemented in the FPGA as a serial port. Nope. Those pins are definitely hard wired the the FTDI USB serial port.

So what else is there? Well, the BIOS is clearly not an RTOS. So we're definitely waiting forever and doing nothing in that while (1) statement. How else does a CPU begin again? Reset of course! Check out the rst implementation in LiteX. It is hooked to R1 of the FPGA. What's that?

R1 (aka BTN_F1) on the ULX3S

That's the F1 button!! (The "Fire 1" position when the ULX3S is in gamer mode! lol). This is also labeled as "B1" on the PCB, located near the SD card, right under the text "2.5V/3.3V"

It turns out the architecture is not [CPU] - [BIOS] - [firmware]. Instead, the simpler: [CPU (with tiny, baked in BIOS] - [BIOS (with external firmware)].  Now, one could certainly write something in the BIOS to, say load an OS. But that's not what LiteX is doing out of the box in this example.

The architecture naming is a bit misleading, as there is part of the CPU implemented as a BIOS as seen at processor boot time on the LiteX. Unfortunately the external "C" program is also called BIOS (firmware).

So just leave litex_term running and press the F1 (CPU reset) button to reload the BIOS firmware. See my soft_cpu.sh bash script. Each time you update and compile the BIOS code, press the F1 button to reload it onto the soft CPU.

I've created LiteX Pull Request #445 that adds some comments to hopefully help others that may encounter these issues.

How many times can an FPGA be reprogrammed? Well, the folks at Numato claim "SRAM based FPGAs can be programmed as many times as necessary. There is no limit..."

There's also the possibility of using a logic analyzer internally to the FPGA. Also mentioned in the What is LiteX is something called litescope. See the Debugging with ChipScope for more information, as there's not much in the Wiki as of the data of this blog. The architecture image provides some insight. See also the test_analyzer.py.

So now.. I wonder if I can single-step code soft CPU code with OpenOCD. And that litescope! That sounds super interesting...

RISC-V on the ULX3S with LiteX - Part 2

First .. the exciting news: With 24 days still left in the Crowd Funding Campaign, the ULX3S is within a few dollars of reaching their $40K stretch goal!! I'm so happy to see their success. I've been super happy with mine. (Edit: now nearly 300% funded by the time I publish this!)

TL;DR: Check out this soft_cpu.sh script used to create a soft RISC-V CPU on the ULX3S.

Back to my fomu! I revisited the Adafruit Feather M0 Express - A request for the USB device descriptor failed: Issue #293 on GitHub in the hathach/tinyusb repo. The suggestion was to simply update my fomu to see if the Feather M0 problems would go away:

The last few lines of the release.sh

dfu-suffix -v 1209 -p 70b1 -a $output/${platform}-updater-${release}.dfu
dfu-suffix -v 1209 -p 70b1 -a $output/${platform}-foboot-${release}.dfu

but I saw an error when trying to do that:

C:\Download\dfu-util>dfu-suffix -v 1209 -p 70b1 -a fomu\hacker-updater-v2.0.3.dfu
dfu-suffix (dfu-util) 0.9

Copyright 2011-2012 Stefan Schmidt, 2013-2014 Tormod Volden
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Please remove existing DFU suffix before adding a new one.

so I next tried using dfu-util, as noted here:

C:\Download\dfu-util>dfu-util -D fomu\hacker-updater-v2.0.3.dfu
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Match vendor ID from file: 1209
Match product ID from file: 70b1
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 1024
Copying data from PC to DFU device
Download        [=========================] 100%       112828 bytes
Download done.
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(8) = dfuMANIFEST-WAIT-RESET, status(0) = No error condition is present
Done!

Since that seems to have been successful, I will return to the ULX3S for now.

A quick check of the rxrbln/picorv32 ULX3S fork that is is included in my full toolchain install and I'm still not yet able to build:

gojimmypi@ubuntu:~/workspace/rxrbln-picorv32/picosoc$ git fetch
gojimmypi@ubuntu:~/workspace/rxrbln-picorv32/picosoc$ git pull
Already up to date.
gojimmypi@ubuntu:~/workspace/rxrbln-picorv32/picosoc$ make
iverilog -s testbench -o hx8kdemo_tb.vvp hx8kdemo_tb.v hx8kdemo.v spimemio.v simpleuart.v picosoc.v ../picorv32.v spiflash.v `yosys-config --datdir/ice40/cells_sim.v`
picosoc.v:70: error: NULL port declarations are not allowed.
Makefile:20: recipe for target 'hx8kdemo_tb.vvp' failed
make: *** [hx8kdemo_tb.vvp] Error 1
gojimmypi@ubuntu:~/workspace/rxrbln-picorv32/picosoc$ 

So ok, I'll give that one some more time.

Refresher on getting Micropython onto the fomu:

cd /mnt/c/workspace
mkdir -p fomu
cd fomu
wget https://github.com/im-tomu/fomu-workshop/raw/master/micropython-fomu.dfu
dfu-util -D micropython-fomu.dfu

Should give an output like this:

dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Match vendor ID from file: 1209
Match product ID from file: 5bf0
Opening DFU capable USB device...
ID 1209:5bf0
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download        [=========================] 100%       136164 bytes
Download done.
state(7) = dfuMANIFEST, status(0) = No error condition is present
state(8) = dfuMANIFEST-WAIT-RESET, status(0) = No error condition is present
Done!

Mine showed up as COM8 and For more details on how a RISC-V soft CPU runs MicroPython, see the Fomu as a CPU section of the fomu workshop.

I've been following along with the development of picosoc for the ULX3S. This is included in the full toolchain build, or can be fetched from here:

cd $WORKSPACE
git clone https://github.com/rxrbln/picorv32.git

To build:

# use the rxrbln firmwware
cd $WORKSPACE/rxrbln-picorv32/picosoc

# there have been some recent additions to the source code; some files not included in the repo
# so get an older, know-to-compile version:
git checkout b1cd395b

make ulx3s_fw.img
$WORKSPACE/ulx3s-examples/bin/ujprog.exe -j FLASH -f 0x200000 ulx3s_fw.img

Use the LiteX CPU

# make the soft CPU
cd $WORKSPACE/litex-boards/litex_boards/targets
./ulx3s.py --device LFE5U-85F --cpu-type picorv32

# show the files built
echo "ULX3S Gateware:"
ls $WORKSPACE/litex-boards/litex_boards/targets/soc_basesoc_ulx3s/gateware -al

echo "ULX3S BIOSL"
ls $WORKSPACE/litex-boards/litex_boards/targets/soc_basesoc_ulx3s/software/bios -al

# put the soft CPU on the ULX3S
cd $WORKSPACE/litex-boards/litex_boards/targets/soc_basesoc_ulx3s/gateware
$WORKSPACE/ulx3s-examples/bin/ujprog.exe top.bit

cd $WORKSPACE/litex-boards/litex_boards/targets/soc_basesoc_ulx3s/software/bios
litex_term --serial-boot --kernel bios.bin /dev/ttyS15

Press enter, and you should see a litex> prompt. Type reboot As a reminder, if you keep getting an error like this:

ULX2S / ULX3S JTAG programmer v 3.0.92 (built Feb 18 2019 10:55:47)
FT_Open() failed
Cannot find JTAG cable.

Check to make sure NOTHING is using the ULX3S, including perhaps a litex_term or putty session. (yes, I've bumped into that more than once).

A big Thank You to @GregDavill  for these tips on the twitter thread:

The LiteX bios is what you're seeing running.  

By default, LiteX builds this to execute from address 0x00000000. This is an address space inside the FPGA, using blockram. Litex embeds the code inside so it's baked into the bit-stream. 

uart_sync() just waits until internal uart FIFOs are cleared. (Which ensures that printf data  has been sent to the PC...) printf also relies on interrupts, so once you've disable interrupts printf no longer works.

The bios exists to initialise things like SDRAM, which you can see it doing here. It then tries to load a USER program from SD/FLASH/Serial/Ethernet.

According to the timvideos "what is litex" the flterm program is what is needed to interact with the bios. As of the date of this blog, I've included that in the full ULX3S toolchain install.

Beware there's an old implementation of traps on riscv32 that may cause code crashes. (e.g. soft debugging)

I found an example using flterm: This next section is an unsuccessful attempt to upload firmware with flterm. (it just sits there waiting for something)

cd $WORKSPACE/flterm
./flterm --port /dev/ttyS15 --kernel $WORKSPACE/rxrbln-picorv32/firmware/firmware.bin --kernel-adr 0x40000000

Searching for the term "liftoff" and I found two occurrences here and here:

# remote.origin.url=https://github.com/enjoy-digital/litex

# this is the one that gets compiled:
C:\workspace\litex\litex\soc\software\bios\boot.c

# plus this older file:
C:\workspace\litex-buildenv\third_party\litex\litex\soc\software\bios\boot.c

I created a soft_cpu.sh script soon to be pushed to the ulx3s-toolchain setup to illustrate how I created a picorv32 soft CPU on the ULX3S using LiteX. See my next blog on updating the firmware.