I was interested in setting up a small Proxmox HA cluster using Ceph to tick the shared data store requirement. I didn’t want used enterprise gear but needed fast networking. The Minisforum MS-01 wasn’t out at the time I was looking but was worth the wait. Anything else would have cost much more and so far they have worked very well for my needs.

Part List

Itemized

Prices are subject to change with sales or things getting old.

Item Price Quantity Total Note
MS-01 13900H Barebone $679 3 $2037 Can not get on amazon
Crucial RAM 96GB $300 3 $900 Not officially supported but works great!
Crucial T500 1TB NVMe $100 3 $300 For Proxmox OS
Samsung PM983a $159 6 $954 For Ceph
USB4 Cables $16 3 $48 Only needed for TB networking
SFP+ DAC $20 3 $60 3m was too long

So all in all the project was looking to cost $4,300 which was a significant investment for a high risk new product with a laptop CPU. However, it feels less expensive in the moment since the cost is distributed across vendors!

Description

Minisforum MS-01

The Minisforum MS-01 was arguably the most important part as it enabled the rest of the spending. I went with the 13900H Barebone model because it was the only one available at the time but I don’t regret it.

The MS-01 was a bit larger than most of the NUCs you’d see but could fit 3 NVMes and a 16 slot x8 speed PCIE card (one slot wide).

ms-01 web

It had two SFP+ ports, two USB4, and two 2.5Gbe which adds up to 65Gbps of networking capabilities (kinda)!

ms-01 ports

The initial goal was to use the 20Gb Thunderbolt network for Ceph’s private network and SFP+ for it’s public plus have other adapters for other things the VMs would require. Ultimately I switched the Ceph private network over to SFP+ because it I wanted more than three nodes and 10Gb was fast enough for my needs.

Crucial DDR5 96GB Laptop RAM

Officially the MS-01 supports 64GB of RAM but strangers on the internet said 96GB would be fine and so far they’ve been dead on. With 20 threads from the i9-13900H I figured more RAM wouldn’t hurt.

crucial 96gb

Crucial T500 1TB Gen4 NVMe

The name of the game for these M.2’s were something cheap that I could install Proxmox on and use for a select few VMs that wouldn’t need to be highly available like k8s nodes which will already re-balance pods if any go down.

crucial t500

Samsung PM983a NVMe PCIe M.2 22110 1.88TB

This was a tricky one but things have been looking good. I wanted an enterprise M.2 with PLP for Ceph since it’s what Ceph recommends and I’m already going through a lot of trouble to make a highly available cluster that it’s not worth saving some money to end up with a corrupt OSD after a power failure.

eBay was the only option for these but everything has been fine so far and I’ve since purchased four more.

Samsung PM983a

Cables

USB4 cables on Amazon are always a risky click unless you go with a brand you can trust. I tent to stick with Anker, Cable Matters, or Monoprice. Ugreen seems to be gaining popularity now for things and I’ve grabbed a few of theirs too.

Now a days I get most of my DAC and ethernet cables from Unifi. I did purchase a 10Gtek DAC when their 3m ones were sold out which was fine but I’m usually on there blowing the budget anyway so I throw in a few cables.

Assembly

Unboxing

I captured enough pictures to “unbox” this live so bare with me. I put together a nice loot pile before starting the lob:

loot pile

The device came nicely packaged in a little cube:

box

Opening it up reveled a nicely wrapped mini workstation:

box open

Next came cracking it open. The lid slid off easily:

slide lid off

Note all your ports say safe, there is no panel here that comes off:

rear ports no lid

RAM

First up was the 96GB Crucial RAM:

ram

This required popping off the CPU cooler fan which had the RAM underneath:

cooler screws

It was a bit awkward to pop off, took a little jiggling, but once off the sockets were easily accessible:

ram sockets

And at the right angle (and flipped correctly) the RAM modules slid in no problem:

ram installed

Then some more jiggling and screwing in the tiny screws got things back together:

cooler back on

NVMes

Next required flipping the device over to get at the three M.2 slots. Not these screws worked better with a screwdriver one size down from the CPU cooler.

nvme cooler

Before sticking in the M.2s I removed a the stand offs that were for shorter M.2s. With a little pressure I could get the screw and standoff out in together. However, I didn’t realize that initially and used some needle nose pliers for the standoffs after removing the screw.

First to go in was the Crucial T500:

t500

Nothing crazy there:

t500 installed

Next were the Samsung PM983a’s for Ceph:

PM983a

Those went in easily as well:

pm983a installed

Finally, securing the rear cooler:

rear cooler back on

GPU (Optional)

I decided to install an Radeon RX 6400 in one of the nodes to get a feel for how it’d perform, fit, and how hot it’d run. Before getting started I noticed I was in grave danger:

beware

The GPU I selected was a XFX RX 6400 which was the only thing I could find that wasn’t VGA port ancient that would fit in the MS-01:

gpu box

Still a monster compared to the little workstation it was going to live inside of:

size comparison

But before it could go in I had to get the half slot faceplate on. This turned out requiring a lot more fiddling than I was use to for network cards and HBAs. The entire cooler and fan had to be unscrewed before you could get at the screws for the faceplate:

cooler off

But things were looking good once it was all back together:

sff

And finally the RX 6400 was snug in the MS-01 and nobody lost a finger!

it fit

First Boot

Now we were all set to boot up and install Proxmox. First boot takes a while so don’t get impatient for it to post:

first boot

I went through the motions of installing Proxmox. Make sure it detects your country and IP/domain automatically or your probably going to have to do an offline install and set up networking later. This happened to me once but rebooting squared it away.

Cluster

One MS-01 gets pretty lonely so it’s recommended you run them in a cluster:

triple boot

In this install I was testing a ring network:

triple boot back

Wires got a bit out of control:

big mess

Finally, they found their home on a shelf in the rack:

in shelf

Next Steps

For what to do now that you have some MS-01’s head over to adding a node.