Turing Pi – The ultimate cluster board for Raspis

Introduction

We use a lot of Raspis in one project of a customer – therefor we need to test it on real hardware. It’s possible to run a pi virtualized to a certain degree with qemu or with a specialized x86 image. But at a certain point you have to run it on real hardware – e.g. when accessing gpio ports. Also we made the experience not everything works flawless on a qemu environment/virtualized environment. So we built a small “cluster” of pis inside of a 19″ rack.

Now we plan to simplify this setup with Turing Pi! And I’m really excited and happy to have finally this board!

What’s a turing pi

A turing pi is a clustering board for 7 pis. This means instead of one pi you can scale up to 7 pis on a single PCB. On the board itself all pis are interconnected via a gigabit switch. On the board itself there’s a 100 mbit uplink. Every node has a seperate gpio header and has it’s own usb port. HDMI is only available for the first node. With a jumper setting the first slot can be used to flash compute modules. Each node has a separate sd card slot if you’d like to use SD cards.

With a turing pi it’s easy for us to scale our test setup – just plug in another Compute Module.

One extraordinary thing to note is the i2c management bus. This bus enables you to turn off/on single nodes with a simple command and more! See https://docs.turingpi.com/turing_pi/children/i2c_cluster_bus/ for details.

What’s a compute module?

To achieve all of this Turing Pi uses compute modules. Compute modules do not have any peripherals on their PCB. Instead they have a pin header similar to SO-DIMM Ram which can be used by breakout boards or in our case the Turing Pi board. Additionally, compute modules can bring their own flash storage directly integrated on the PCB. This is handy if you don’t like to handle SD cards.

You can easily identify whether your compute module does have integrated storage or not:

You see there’s a chip missing on the left compute module. That’s the one without integrated storage. If you’re going to use compute modules with integrated storage use one with enough storage – it’s obvious that you cannot easily update the storage as with an SD card.

We’ll use compute modules with integrated storage – it’s more convenient for us if we’d like to add additional nodes and currently I don’t have enough SD cards in storage. But it’s only a small improvement in handling – inserting an SD card is not a big deal with the Turing Pi.

What’s in the box?

The PCB comes in a plain box without any peripherals. It’s good secured with some foam and the PCB itself is inside in an anti static foil. I think it’s a good idea to deliver the board without any additional hardware like SD cards or even compute modules. Often we order Pis and have a lot of additional power supplies or small SD cards in storage which we’ll probably never use. We prefer to buy only what’s necessary – but that’s not always possible.

Wiring

Wiring the Turing Pi is really straight forward. For a first test run you need at least to wire power and network – the backup battery for the RTC clock can be omitted for first tests. Power can be supplied either with a barrel connector or an 4 pin ATX mainboard power supply. Be aware, that according to specs that with a fully populated Turing Pi 5A max current is flowing with 12v voltage. Your power supply should be able to provide enough Amps. Most probably the wires for barrel connectors are too small for a fully populated board. But for first evaluations we just used a simple barrel connector for power supply – but the board is not fully populated.

Later on we switched to a DIN Rail power supply with 5A max. For a secure connection between power supply and the board we’re using the 4 Pin connector (2 +12v, 2 Ground wires) with thicker wires. This should be safe enough. Now just plugin a patch cable and if you like connect the HDMI port to a monitor. Insert one compute module in the first slot and power it on.

Some LEDs should light up – if not check the voltage of your power supply with a Multimeter. Mine was set too low and had to be increased via the Potentiometer.

Setting up first node – flashing via usb

When using a compute module with integrated storage the device needs tob e flashed via USB. The Turing Pi Board uses a separate usb micro port which is wired to the first port. When flashing set the jumper accordingly:

On your workstation you have to install usbboot https://github.com/raspberrypi/usbboot . The readme should contain all required steps.

Important: Use a “good” usb cable. I wasted several hours around why the device was not recognized. The problem was a faulty usb cable which works fine for charging my phone but doesn’t work when using for data transmission.

Steps with a Linux maschine:
0.) Install usbboot
1.) Set Jumpers to flash and connect your usb cable

2.) Execute:
sudo ./rpiboot
3.) Wait til it’s finished with copyig files

4.) Check that new device is visible with lsblk
-> Should be something like sda, sdb etc.
-> This device is the storage on the compute moduleSee https://docs.turingpi.com/turing_pi/children/i2c_cluster_bus/ for details.
5.) Copy raspbian with dd
Be sure that your writing to the device and not your hard disk. If dd’ing exits successfully after some seconds, you’d just copied the image file and not written to the device(been there:))
dd if=raspbian-buster-lite.img of=/dev/sdX bs=4MiB
-> This takes about 5 to 8 minutes on my machine
6.) Put the jumper back, disconnect USB and powercycle the board. Raspbian should now be booting up

When it’s not working/dd hangs: Ensure that there’s no file called /dev/sdX. Check it with ls before trying again

If your using OSX – have a look at Jeff Geerling’s blog article – https://www.jeffgeerling.com/blog/2020/flashing-raspberry-pi-compute-module-on-macos-usbboot

I’ve written a little script to flash all compute modules one after the another(needs root rights – use sudo). So you just have to plugin/unplug the Compute Module into the first slot.

Netbooting compute module

Sadly my compute modules hadn’t netbooting enabled by default. In our project netbooting is a integral part. After flashing and booting up the pis we applied the following ansible playbook for enabling netbooting:

Be aware that this playbook reboots the pi and edits /boot/config.txt. I know it’s not the cleanest one(using sudo etc) but for this one shot task it’s good enough. But use at your own risk 🙂

You can verify that the otp flags are written successfully with:

This should return:
17:3020000a


For setting up a netbooting capable server – have a look at https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net_tutorial.md
There are a lot of good tutorials how to do it – It’s not a good idea to write another tutorial 🙂 Stick with the official documentation.

Pros and Cons compared with the old setup

To be honest – it’s really difficult to see any cons compared to the old setup. Wiring is simplified enormously, scaling up is easier, resetting of nodes is simplified, RTC clock onboard, better overall network speed(not benchmarked – just an impression), no PoE enabled switch is needed, less wires, smaller form factor.

One disadvantage is the use of compute modules instead of “standard” raspis. What’s a little bit annoying that netbooting is not enabled by default which means you have to flash every node by hand once and set the otp flags. Also only one HDMI port for the first node could be in certain setups a little disadvantage. But to be honest, these cons are very, very little compared to the gained advantages.

Some thanks etc

First of all I’d like to thanks Costa from Turing Pi for answering and handling my “special” shipping request. And most of all the TuringPi Team for designing, developing and all things around to build a really cool product! I can’t imagine how much work is behind this product!

Not to forget Thomas Kraft for the really fast support in buying the required compute modules 🙂

Where to go next?

Currently TuringPi is sold out but you can register to get a notification when the next batch is planned to be produced. TuringPi provided some documentation how to setup a kubernetes cluster with a Turing Pi.

What’s coming in the second part

In the second part of this post we’re gonna show how we use the Turing Pi in combination with gitlab ci and ansible to test our ansible scripts. Stay tuned!