Forming a storage cluster while moving into another new flat

I am once again moving into a new flat. I have setup a special renting contract for only 6 months (which is very hard to do in austria and probably illegal). I do not plan to stay in this small flat for any longer as the flat that I have already bought is being finished soon. More on that in a future post.

This new flat needs give my home network infrastructure its own room again. The storage room has neither a power outlet nor a network connector box. We will wire both randomly on the heating pipes.

Also I need to increase my storage capacity once again but I do not want to put more hard disks on the USB3 bus.

ODROID-HC2 cluster!

Hardkernel just released a new type odroids (ODROID-HC2) which are made to be stacked on each other. They are powered with a 12V power supply

Having multiple hosts on the network is far more scalable on the long run, as we can always buy more and larger switches and more storage nodes. We could of course continue to use BTRFS raid and connect all the disks over NBD to a single main server, but this will not be very reliable. What we need is a failure tolerant distributed cluster file system. For my first testings I chose Google’s glusterfs on archlinux arm.

Putting my old homeserver togheter with the first odroid-hc2 storage nodes looks pretty nice:

Acquiring a new switch

As you can already see in the video, my single small switch is not very suitable anymore, it’s about time to get a first professional grade network switch.

Ubiquiti’s 48 port Gigabit Ethernet Switch enters the room!

It also has 2 10Gb/s SFP+ ports for future expansion. Those switches are not cheap (~400€ at the time of buying).

I have had 10 Raspberry Pi 3 lying around from an final apprenticeship examination project which finally also find some space! Using glusterfs these will all be able to mount my new distributed filesystem and access different files concurrently without filling up only one network link. Reading and writing files will require them to talk directly to the right storage node. The switch has a internal switching bandwidth of around ~70Gb/s

Let’s see what things I can do now with my increased storage and processing capacity.

I think that if my server infrastructure keeps growing like that, I will need a proper server rack in the near future.


Posted

in

by

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *