A RISC'y cluster - Part II

Building a chassis for a 19" 1U four node cluster of StarFive VisionFive2 boards.

Getting the boards set up with a clean Debian Trixie install on on-board NVME storage was treated in the previous post. Now we need to create an enclosure that will fit four of these boards in a single unit of a shallow 19" wall-mounted rack.

Why the dimensions?

First of all, organising computers and switches and the like in a 19" rack is just convenient. My patch panel and my switch are already there, I would need very good arguments for doing anything other than that. However, the wall mounted rack is just a couple of pieces of mild steel flat stock I had lying around that I drilled, tapped, bent and welded - it’s a 19" rack with all the proper spacings between the rails and the holes, but it is much too shallow to fit any regular 1U rack server. Therefore, the cluster enclosure should not be deeper than a regular switch.

Power supply

The VisionFive2 board is powered via USB-C PD. According to the schematics (which SiFive - to their tremendous credit - release and publish), it is actually safe to ignore the PD part and simply deliver up to around 20V directly on the USB-C power pins. The board has on-board power regulators (Buck converters) that will convert this down to whatever is needed - and actually if PD is implemented on the USB power supply side the board is going to ask for about that amount of voltage anyway.

This leaves two good options for powering the board:

• either get a standard USB-C power brick with PD support and around 30W capacity
• or, build a crude 15-20V power supply that simply feeds this voltage directly to the USB-C power input on the board (and make a note not to connect any other USB-C devices to that power supply)

I really wanted to go with the latter approach - building a compact switch mode power supply custom for the project is such a cool idea. Designing the chassis around a common (or dual) power supply module would lend itself well to a common airflow as well (so single or dual fan instead of per-node fans), and of course one could imagine a power back-plane so that there would be no wires or plugs, just simply plugging in the node into the chassis would ensure it is powered.

But then, I need to have this done and working. This time around I opted for simply buying a handful of simple individual USB-C PD power bricks and powering each board from its own brick. (In my experience these bricks have a not insignificant failure rate, so it makes sense to both keep a spare, and to power the boards individually from each their own brick, so only one node fails at a time).

Overall chassis manufacturing

I started drawing up a chassis where I used sheet metal for the top and bottom, while 3d printing the sides, front and back. However, after working with this for a while and looking at other peoples projects (like this one), I decided the whole thing could be 3d printed.

A couple of things I wanted:

• Must fit in a single unit in a shallow 19" rack
• Individual nodes should be able to be pulled out of the chassis
• Easy access to network ports on the front
• Design must allow for active cooling

With an Ender 3 V3 SE and a print bed size of 220mm squared, clearly the chassis will need to be assembled from multiple components.

The other chassis (linked to above) use threaded rods transversely to tie the individual chassis parts together, however I could not make this work in my design while also allowing for active cooling (at least not with a reasonably sized fan). I ended up designing for gluing the permanent joints, and snap-fitting the parts that may need to come apart again.

Having bitten the bullet of combining individual components, it made sense to also design for support-less printing. Everything prints without the need for supports. The only post-processing necessary in the current design, is the threading of a couple of M3 holes (for mounting a fan and the SBC itself) in each board carrier.

The whole thing prints well in PLA; there should be no need for higher temperature tolerance, there is no need for tolerance towards exposure to sunlight or the elements (I hope), and PLA is both easy to print, cheap and very strong.

The “shells”

The chassis is made up of four shells - one shell for each node in the cluster. A shell is printed in two halves with appropriate dowels and holes that make them locate together accurately for an easy fit-up when gluing the halves together.

The large horizontal cylindrical hole at the rear is for the USB-C power cable which need to come in from the rear side in order to allow pulling out a node carrier from the front of the rack.

The rectangular hole next to it is where the “fan tower” in the board module slides in. Air will be extracted this way out the back of the chassis, with air intake through the front face plate of the module.

A printed set of half-shells are simply glued together by sticking one side on top of the other after applying glue.

The tabs and indents on the side allow the assembled shells to locate neatly together after having applied glue on the sides. Of course the “ears” or sides of the chassis will form the outer extremes. This is the final step in assembling the chassis proper.

The “ears”

For mounting the chassis in a rack, we need “ears” on the chassis. These ears also form part of the side of the chassis, and will be glued on to the sides of the chassis.

As can be seen, the ears have the same locating tabs and indents as the half-shells, and they do indeed fit on the sides of the chassis.

The carrier module

The actual SBCs, the cluster nodes, will need to be mounted on a “carrier” module which will also hold the fan for active cooling.

This is by far the most complex part, and the most fragile. At the very rear it has a “fan tower” for holding a 40x40x20 cooling fan. A duct pulls air from both above and below the board which is mounted on little posts (with M3 threaded holes that need to be manually tapped).

Holes in the back allow for the USB-C power connector and for access to the reset button.

The thin side posts allow for snap-fitting a front plate. Once the front plate is fitted, the module is quite rigid and strong, but clearly the design with the tall thin walls near the front are a weak point on this design prior to (and during) assembly.

The face plate

The module will have a face plate that slides on to the front of the board (sliding over the Audio, USB, HDMI and Ethernet connectors). It will have holes to allow air intake both below the board (for airflow over the NVME module) and above the board (for all the top mounted components, the CPU in particular).

Two little tabs allow snapping on to the thin sidewalls on the carrier module. Two little tabs in the handle cutouts allow locating the bottom of the face plate. Together, while finicky to assemble, the module and face plate hold nicely together with the board holding the face plate down.

Printing

Get the STLs here

• Shell
• Ears
• Carrier module
• Face plate

Shell, carrier module and face plate must all be printed a total of four times. For me, PLA worked perfectly. They should all print just fine without supports.

I printed the chassis, ears and face plates in black, and the carrier modules in yellow. Looking at the assembled system from the front, only the handle part of the carrier module is visible, and I rather like the bright yellow handle against the black backdrop of the face plate. Pick any combination you like of course - you can get any two-colour combination you want even with a single colour printer.

Assembly

Major parts are glued. I used a simple water soluble glue (Pattex “no more nails”) which took forever to harden, not the best choice I’m sure. If you are not used to gluing 3d printed parts together, I highly recommend reading this guide to gluing prints, written by the good folks over at Prusa Research.

The chassis

First each of the four shells should be assembled by gluing the two half-shells together. Let the glue harden on the four shells before attempting to combine them.

Second, the four shells and the ears can be glued together. The pins and tabs should locate neatly together (test fit before applying glue). You probably want to place the whole thing on a very flat surface, put some weight on top of the modules (to hold them down against flat), and to clamp the sides together so the glue will get a good bond and the whole thing is held flat, square and tightly together. I used a large F-clamp (with some wooden blocks to distribute the pressure evenly onto the sides) which would span the near half meter width of the chassis.

Once cured, test fit the chassis in your rack.

My chassis is mounted on top of a switch, which means, it is supported on the bottom. I don’t believe support on the bottom should be necessary - a glued bond with these large areas should be very strong. However, it is probably unwise to have the chassis sit unsupported in the rack and then place heavy objects on top of it. Common sense applies.

The nodes

Tap at least a few of the holes on the carrier module to be able to bolt the board down. I recommend tapping at least one of the holes near the face plate (as the board holds the face plate down).

Now first press the face plate on to the VisionFive2 board. It’s a pretty tight fit (if needed, sand, file or cut the holes to accommodate the connectors - mine fit without modifications though). Don’t worry about the board carrier module at this point, only focus on getting the face plate on to the VisionFive2 board.

Once the face plate is fitted on to the board, we can go ahead with what is the only really tricky part of the assembly. We now need to accomplish a couple of things in order:

• The rear of the board must be located on top of the posts. It must then slide backwards so that the USB-C connector and reset switch mate up to the holes in the rear of the carrier module.
• The face plate must mate up against the thin sidewalls on the carrier module and then be pressed down so that the little tabs on the bottom slide in to the indents at the handle

Basically the front panel tabs will “click” into place in the handle indents, while the tabs on the thin sidewalls on the module fit into the little tabs on the face plate. This is finicky. I can highly recommend practising this a few times without the actual VisionFive2 board. Just clicking in the face plate. It’s a lot more tricky with the board on. Maybe print an extra module and face plate so you don’t have to worry about breaking the first set.

Be sure the USB-C connector protrudes into the cutout for it (this actually holds down the board on the rear, making fasteners an optional extra. There is a little bit of give in the PLA, but if the USB-C connector is not fitted up against its cutout, assembly will be very difficult.

Now, finally, add at least one screw to hold down the front of the board, since this is what holds the faceplate down in its place. I used Nylon bolts for bolting down the board - metal bolts will work but it is not possible to torque them down sufficiently for them to not come lose. I guess it doesn’t matter very much for holding down the board. Something like this will do the trick.

Now all that is left is to connect the fan. I bought a couple of different types of fans, none with connectors that match the board. The board is using a JST-PH connector for its fan header. The board supplies 5V to the fan and it is always on with no means of controlling or switching the output.

I only used two of the total of four possible screws to hold down the fan - this is more than enough and the fan is not visible from the front anyway.

Final assembly

Four of these modules can now slide into the chassis for a neat little four node cluster. Don’t forget to feed the USB-C cables from the power bricks through the holes from the back and out through the front of the chassis. This allows for easily connecting the power to the finished node module and then inserting the module into the front of the chassis (letting the power cable slide back out through the hole as needed). When tidying up the cables in the end, remember to leave some slack on the cables so you can actually pull out your modules, should you need to.

With this, I have four neat little identical computers humming along, all connected to gigabit Ethernet. All four nodes run a standard Debian Trixie install. What I need now is services on top.

If you’re trying to replicate this setup and run into trouble, feel free to send me an email - I can’t promise that I can solve your problems but I can at least give you a reply.

P.S. The VisionFive2 Lite

Of course, basically as this project completed, I learned about the kick-starter on a VisionFive2 Lite board. This board would have let me fit seven nodes instead of just four in the same space, with practically the same specs (only the CPU would have been slightly slower).

I don’t know if I would have made a 7 node cluster on that board though. No matter how many nodes you have, the JH7110 CPU is not a speed daemon - a large cluster of these does not make sense from a compute power point of view. As far as redundancy goes, a four node cluster is already extravagant. I guess you could host two three-node clusters and use the remaining space for a central power and cooling module - that would actually be a cool project. Maybe another time.