We’ve NEVER done this before… - Mother Vault Part 1 - JBOD

Linus Tech Tips2 minutes read

The Vault system recovered most data from 2.4 petabytes of storage, transitioning to the new Mother Vault solution with three massive servers for improved capacity and efficiency. The JBOD configuration using Super Micro 947 HE1C-42K 05 JBOD with dual AMD EPYC Milan processors manages 70 drives in a RAID Z2 configuration but plans to switch to 10-drive RAID Z2s for increased speed.

Insights

  • The Vault, an archival storage system, faced challenges but successfully recovered data from 2.4 petabytes of storage, now distributed across five aging servers.
  • Transitioning to the Mother Vault solution involves deploying three massive servers, each capable of storing nearly two petabytes, aiming to streamline data management and increase storage capacity efficiently.

Get key ideas from YouTube videos. It’s free

Recent questions

  • What issues did the Vault archival storage system face?

    The Vault faced issues but managed to recover most of the data from 2.4 petabytes of raw storage.

  • What is the new solution to the archival storage system?

    A new solution, the Mother Vault, will involve deploying three massive servers, each capable of housing nearly two petabytes of hard drive storage.

  • What was the original petabyte project in 2015?

    The original petabyte project in 2015 consisted of two 45-drive storinator servers holding 120 10TB hard drives, totaling 1.2 petabytes raw.

  • What configuration was chosen for managing multiple clusters?

    Transitioning to a JBOD configuration was chosen for its simplicity and reliability.

  • How is the JBOD chassis configured for data management?

    The JBOD chassis contains no computer, only SAS components, with hot-swappable SAS expanders capable of connecting up to 30 drives each.

Related videos

Summary

00:00

Efficient Data Storage Solution: The Mother Vault

  • The archival storage system, known as the Vault, faced issues but managed to recover most of the data from 2.4 petabytes of raw storage.
  • The recovered data is now spread across five servers, each capable of holding close to two petabytes, with some servers nearing a decade in age.
  • A new solution, the Mother Vault, will involve deploying three massive servers, each capable of housing nearly two petabytes of hard drive storage.
  • The original petabyte project in 2015 consisted of two 45-drive storinator servers holding 120 10TB hard drives, totaling 1.2 petabytes raw.
  • A second petabyte project was built later, using 75 16TB drives across two more 45-drive storinators, creating two separate network storage shares.
  • Managing multiple clusters became inefficient, requiring significant time and hardware components for maintenance and expansion.
  • Transitioning to a JBOD configuration, specifically the Super Micro 947 HE1C-42K 05 JBOD, was chosen for its simplicity and reliability.
  • The JBOD chassis contains no computer, only SAS components, with hot-swappable SAS expanders capable of connecting up to 30 drives each.
  • A head or controller server, connected to up to 270 drives, will manage the JBOD setup, utilizing Broadcom HBAs for data transfer.
  • The controller server will feature dual AMD EPYC Milan processors, around a terabyte of RAM, and potential NVMe drives, optimized for ZFS compression and data caching.

15:00

"JBOD Configuration and Performance Optimization Strategies"

  • The array consists of about 70 drives, set up as four V devs in a RAID Z2 configuration with 15 drives wide, impacting performance.
  • Plans to switch to 10-drive RAID Z2s for increased speed are in place, despite not expecting significant performance improvements.
  • External mini SAS HD cables from Infinite Cables connect the server to the JBOD, which is initially configured as a single zone.
  • Zoning options allow for dividing the JBOD into two or three zones, each serving different controller servers with limited data access.
  • The JBOD's power supplies, at full capacity, are extremely loud, prompting consideration of rewiring the server room for 208 volts.
  • The JBOD's configuration can be adjusted for single, two, or three zones, enhancing throughput and redundancy.
  • The data set is set up with caching only metadata, with RAM cache turned off, resulting in lower sustained data transfer speeds.
  • Maintenance benefits include easy drive replacement due to a ribbon cable system, eliminating the need to remove all cables for access.
Channel avatarChannel avatarChannel avatarChannel avatarChannel avatar

Try it yourself — It’s free.