Phishman – Tech

After doing extensive research on single board ARM-based computers I decided to build my own cluster. The purpose of the ultra low power cluster is to develop new applications with the most recent software technologies and host few production websites. Yes, I know, I’m crazy! 😀

My research showed that previous clusters lack significant calculating power and they’re based on performance limited (hardware) boards like Raspberry Pi 2, Raspberry Pi 3 and PINE A64+. The main problems with those boards are the RAM, the limited I/O of the microSD cards (limited to 25 MB/s on Pi 2 and Pi 3), and the network speed (10/100).
Those boards are more suitable to be used as development boards. Often microSD card speeds are limited to 25MB/s (on Pi 3), which means that even if you buy UHS-II and UHS-3 cards you won’t be able to utilize them at their full speeds.

I’ve looked closely at the ODROID single boards – those have powerful CPU (Quad and Octa-core A53 and A15), fast Gigabit network (10/100/1000) , 2GB RAM USB 3.0, eMMC support.

For my build I’ve chosen the following hardware

Parts and single-boards – www.ameridroid.com

Web and Caching Nodes:
ODROID-C2:

Web Server Nodes:
ODROID-XU4:

We benefit from faster more capable CPU, eMMC (up to 400MB/s capable eMMC5.0) speeds, USB 3.0, Gigabit network and 2 GB of LPDDR3.

Database Nodes:
ASUS H110S2/CSM Mini-STX (Installing Percona With TokuDB MySQL Server)

For our cluster we will start with:

• 1 proxy node
• 2 load balancer nodes
• 2 web server nodes
• 1 database server node

Our final goal is a cluster with:

• 1 proxy node
• 2 cache nodes
• 8 nodes in web cluster
• 10 nodes in MySQL cluster and 2 nodes in NAS cluster. Total 23 nodes.

We may not be able to fit that many nodes in this printer case, because the power supplies needed to power all the nodes will require a lot of space.

Hardware Prices:
$90.75 – 2 x ODROID C2
$53.90 – 2 x 16GB eMMC Black Module C2 Linux
$133.05- 2 x ODROID-XU4Q with Passive Heatsink
$54.90 – 2 x 16GB eMMC 5.0 Module XU3/XU4 Linux (Blue Dot)
$20.15 – 2 x 5V/4A US Type Power Supply
$30.00 – 1 x Raspberry Pi B+
$39.00 – 3 x MicroSD Sandisk 16 GB UHS-I
$3 – 1 x Display flat cable 22 pin to 15 pin Pi (China)
$29.99 – 20 x 3 ft CAT6 patch cable – MAXIMM 3 Feet |20 Pack | Red| Snagless Cat6 Ethernet Cable With Cable Ties
$15 – 1 x SABRENT SSD Enclosure USB 3.0 EC-UASP
$20 – 2 x SABRENT SSD Enclosure USB 3.0 EC-UK30
$14.99 – 1 x KEEPING 5V/12A 5-Port USB Hub 60W Wall Power Charger Sation for Tablet, iPhone,Samsung Galaxy, Amazon Kindles, HTC, LG
$100 – 2 x SSD PNY 120 GB
$74.36 – 1 x NETGEAR 24-Port Gigabit Ethernet Desktop/Rackmount Switch (GS324)
$8.63 – 1 x M3 25+6mm Male Female Thread Nylon Hex Standoff Spacer Pillar 50pcs
$8.90 – 1 x HOBBYMATE 180Pcs Assorted M3 Nylon Screws Nut Standoffs Spacers for DIY
$5.89 – 1 x Orion 3d Black Carbon Fiber Vinyl Wrap – Outdoor Rated for Automotive Use – 12″ x 60″
$5.90 – 1 x Pwr+ Short 3 Ft 3 Prong AC Power Cord Cable Plug L-Type
$1.00 – 1 x Mesh organizer
$81.49 – 1 x ASUS H110S2/CSM LGA1151 DDR4 Dual GB LAN DP HDMI LVDS Mini-STX
$50.00 – 1 x Intel Pentium G4400 SkyLake 3.3 GHz LGA 1151 Boxed Processor (for BIOS update)
$50.00 – 1 x Intel Pentium G4560 Kaby Lake 3.50GHz LGA 1151
$216.98- 1 x G.SKILL 32GB (2 x 16G) Ripjaws Series DDR4 PC4-17000 2133MHz Laptop Memory Model F4-2133C15D-32GRS
$135.00- 1 x Samsung 960 EVO Series 250GB NVMe M.2 Internal SSD
$26.56 – 1 x Oneda 120W 19V 6.32A AC Power Adapter Laptop Charger for Asus
$14.99 – 1 x Cable Matters (2-Pack) 4-Outlet AC Power Cord Y-Splitter – 1.5 Feet
$7.99 – 1 x HUACAM HC05 3 Pack USB to 5.5 mm/2.1 mm 5 Volt DC Barrel Jack Power Cable
TOTAL: $1292.29

Case enclosure used is old broken Canon printer, which I liked for the project.

Building

I was installing TokuDB Storage Engine on Percona 5.7 Server based on this Percona guide, when I realized that there’s no my.cnf file.

You are still able to configure the server and install plugins from within the MySQL CLI.

Tech Specs:
OS: Debian Jessie
RAM: 32 GB
CPU 2 Cores, 4 Threads
Storage: SSD NVMe Samsung 960 Evo

More about the server:
This little MONSTER is amazing!

I have it paired with Intel G4560, 32 GB DDR4 G.Skill Ripjaws (2 x 16 GB) and M.2 Samsung Evo 960 250GB. Installed Debian Jessie server edition (with latest kernel) and did SSD read/write benchmark. It does 1.7 GB/s write and 3.8 GB/s read of 1 GB file. Insane!!!
Stock CPU fan – CPU 32C, motherboard 34C degrees. Under load 38-40C max!

I had to buy Skylake CPU (Intel G4400) to update the BIOS for the Kaby Lake (Intel G4560) support. It’s LAME! Those manufacturers must integrate USB flashing by default as a basic functionality! If I had time I would send the motherboard back or just return it. Unfortunately I had no time to ship my motherboard for BIOS update. It was in the product description that it supports 6th Gen Skylake CPU, even on ASUS’ website it was clearly stated that it needs BIOS update, so it’s not something that I didn’t know. It still is the best, smallest STX motherboard!

I would highly recommend and I would buy AGAIN this little monster board! I hope it stays sharp and fast!
I’ve been waiting for an STX form factor for years. It’s dream come true! 🙂

Thanks crazy awesome ASUS! 🙂

Here’s how to do it:

1. Installing Percona Server on Debian and Ubuntu (Official)
2. To avoid “ERROR 1123 (HY000): Can’t initialize function ‘TokuDB’; Plugin initialization function failed”, you need to disable Kernel Transparent Hugepages for TokuDB, if not disabled.
   Here’s how to check:
   
   Check Transparent Hugepages Status

   cat /sys/kernel/mm/transparent_hugepage/enabled

   1	cat /sys/kernel/mm/transparent_hugepage/enabled

   
   It should return something like:
   
   [always] madvise never

   1	[always] madvise never

   
   If it’s not set to never, you can do:
   
   Change Transparent Hugepages in The Kernel Config

   if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi

   1 2 3

   if test -f /sys/kernel/mm/transparent_hugepage/enabled; then    echo never > /sys/kernel/mm/transparent_hugepage/enabled fi

   
   Reboot.
   Test again after boot:
   
   Check Transparent Hugepages in Linux Kernel

   cat /sys/kernel/mm/transparent_hugepage/enabled

   1	cat /sys/kernel/mm/transparent_hugepage/enabled

   
   Now it should be:
   
   always madvise [never]

   1	always madvise [never]

   
    
3. Dependencies
   You need libjemalloc library installed. Quick and simple installation:
   
   Install libjemalloc And Unmet Dependencies

   apt-get install libjemalloc-dev apt-get build-dep libjemalloc-dev

   1 2	apt-get install libjemalloc-dev apt-get build-dep libjemalloc-dev

   
    
4. Install The TokuDB Engine
   
   Install The TokuDB Engine and Plugins

   mysql -uroot -p # Install the TokuDB Engine mysql> install plugin tokudb soname 'ha_tokudb.so'; # Install the Other TokuDB Plugins mysql> install plugin tokudb_trx soname 'ha_tokudb.so'; mysql> install plugin tokudb_locks soname 'ha_tokudb.so'; mysql> install plugin tokudb_lock_waits soname 'ha_tokudb.so'; mysql> install plugin tokudb_file_map soname 'ha_tokudb.so'; mysql> install plugin tokudb_fractal_tree_info soname 'ha_tokudb.so'; mysql> install plugin tokudb_fractal_tree_block_map soname 'ha_tokudb.so';

   1 2 3 4 5 6 7 8 9 10 11 12

   mysql -uroot -p   # Install the TokuDB Engine mysql> install plugin tokudb soname 'ha_tokudb.so';   # Install the Other TokuDB Plugins mysql> install plugin tokudb_trx soname 'ha_tokudb.so'; mysql> install plugin tokudb_locks soname 'ha_tokudb.so'; mysql> install plugin tokudb_lock_waits soname 'ha_tokudb.so'; mysql> install plugin tokudb_file_map soname 'ha_tokudb.so'; mysql> install plugin tokudb_fractal_tree_info soname 'ha_tokudb.so'; mysql> install plugin tokudb_fractal_tree_block_map soname 'ha_tokudb.so';

   
   Test to check, if the TokuDB storage engine was successfully installed do:
   
   View Installed MySQL Engines

   mysql>SHOW ENGINES;

   1	mysql>SHOW ENGINES;

   
    

That’s it. Enjoy! 🙂

Feel free to shoot me a message in the comments.