I present Odin, the 60TB Plex Media Server

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
So, as a gift to myself for my Birthday, and for paying off my Credit Cards I decided to use my Tax money to upgrade my FreeNAS server into something truly special [/Snoke Voice]

One of the problems I've had with it wasn't as much storage anymore, but the ability to expand or perform drive maintenance. Desktop Cases just aren't designed to be servers and its hard/annoying to get at the drives. And there isn't enough room for drive expansion later on.

So I went and decided to purchase a bigger case. I looked around at Desktop cases, even full towers. But then someone pointed out the NORCO RPC-4224 server chassis which works with ATX style components. It offers 24 3.5" hot-swap bays connected via 6 SFF-8087 SAS3 backplanes. Each backplane supports SAS3 which gives 12Gbits/s throughput for each drive.

To connect those drives I needed SAS controller cards. So I bought used SAS controllers (cause these things are EXPENSIVE).
I ended up with a SAS 9201-16i which supports 16 drives via 4 SFF cables. I needed another two so I bought an additional SAS 9211-8i controller. So now all 6 backplanes are functional.

To keep cables need and tidy I bought right angle SAS cables in the shortest length I could, but thats .75 Meters, which is still too long. However, without custom cables that will have to do.

I also didn't want a Rats nest of power cables I wasn't using so decided to change out to a fully modular power supply. I bought a Corsair RM750x modular power supply so I could install only the cables I need.

When I first got the case, I found that while it wasn't what I would call Server Loud, it wasn't quiet enough for me. So I bought 3x 120mm Noctua PWM fans, 2x 80mm Noctua PWM fans, and a Noctua U9S heat sink cooler which is very quiet and very efficient. It also fits nicely in the 4U chassis.

The parts list to build this server is below. Some of the parts are inherited from the previous desktop case. Some I bought new to make it work in the new chassis.

  • NORCO RPC-4224 24 3.5" bay Hot-Swappable 4U Server Chassis
  • Corsair RM750x Modular power supply
  • Noctua NH-U9S 92mm Silent CPU HSF
  • Intel i7-4790k Socket 1150 CPU
  • Gigabyte GA-Z97X-SLI Motherboard
  • 32GB Crucial Ballistics Sport DDR3-1866mhz ram
  • 6x right angle SFF-8087 to Straight boot cables
  • LSI SAS 9211-8i Host Bus Adapter Card
  • LSI SAS 9201-16i Host Bus Adapter Card
  • 3x Noctua NF-S12A 120mm PWM fans
  • 2x Noctua AF8 80mm PWM fans
  • 6x WD Red 8TB drives (32TB RaidZ1 Array)
  • 5x WD Green 4TB drives (16TB RaidZ1 Array) WDIDLE'd
  • 1x Sandisk X400 128GB SSD (FreeNAS OS Boot Drive)
  • 1x Samsung 850 EVO 256GB SSD (Plex VM and Plex Database)
 
Last edited:

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
On to the pictures of the build!

So, the first thing I did after receiving the chassis is build a sacrificial NAS using old WD Red 3TB drives that I didn't care if they blew up.

I wanted to test all of the Backplanes and ports to make sure it was all working.

Some Parts have arrived.

27987642949_65ed1a5f42_o.jpg


39056430504_ebfcfcb3c8_o.jpg


39134496954_b92ceed960_o.jpg


I got it up and running with an old AMD CPU/Mobo, and tested each backplane 2 at a time.

38947640795_2ce43887aa_o.jpg


Taking a look inside one of the Hot-Swap bays and its hard drive caddies.

39136679244_7ae9827ee0_o.jpg


39136679414_d34ac86ca1_o.jpg


25973307668_4a6c3fe94a_o.jpg


After I confirmed everything was working it became time to strip the chassis back down and migrate all the Good stuff to the new chassis.

But first the Cats needed to inspect it.

38947640745_d26ff93413_o.jpg


24975920847_71ca4500e8_o.jpg


25973308178_657d1eb2d3_o.jpg


The Cats didn't approve of the noisy stock fans. So out they went, and in went the Noctua silent PWM fans. Starting to add components here.

The internal 2 bay 2.5" HD tray.
28071202959_e347759ecf_o.jpg


The Noctua NH-U9S 92mm HSF, which is so very quiet. And the CPU temps are staying right around 37°C idle.

39850483921_648bf3127a_o.jpg


Starting to install the components.

28071202779_92a3bc8c42_o.jpg


Cable Managing EVERYTHING from the get go.

28071202719_dcf6a2a9b4_o.jpg


39850483301_5c62befb70_o.jpg


Adding in the SAS cards and SAS cables. They are too long, so they make the biggest mess in the case. But I did a decent job with them I think. They are really rigid and difficult to work with. The right angle boots really make the drive cage side super clean and easy.

39850483041_34a1f0fd71_o.jpg

39140345054_1f3e8a443a_o.jpg

39140344204_5e8bf12994_o.jpg

39140344484_e093fd4e17_o.jpg


Running power to the backplanes.
39140344054_f9f162acf0_o.jpg

39850482841_33c09726f5_o.jpg

39850482611_c93394a27b_o.jpg




And finally, to aid in keeping the drives cool I installed them staggered for now to increase airflow around them. Pretty blue LEDs!
38951978715_1d4c677de1_o.jpg


And here we are, 54TB of formatted storage.

I named the server Asgard, because Thor: Ragnarok. Inside of Asgard live Odin, and Frejya.

Code:
[william@Asgard ~]$ zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
Freyja        18.1T   824K  18.1T         -     0%     0%  1.00x  ONLINE  /mnt
Odin          36.2T  17.4T  18.8T         -    11%    48%  1.00x  ONLINE  /mnt
freenas-boot   118G   840M   117G         -      -     0%  1.00x  ONLINE  -
jails          230G  63.2G   167G         -    37%    27%  1.00x  ONLINE  /mnt
[william@Asgard ~]$


How does 54TB of formatted storage sound? The shear quietness of this server astounds me. It is literally WHISPER quiet. It doesn't even register on the Decible meter on my iPhone over ambient background noise. So awesome.

It sounds like SILENCE! This thing is so quiet I can hear my Pet Fountain and refrigerator over it. its just SOOOO quiet. Was well worth the money spent on the Noctua Fans and Heat Sink combo.
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
Build pictures all posted now.

This is awesome! Did you do this all for yourself, or are you running your own subscription service?!?! lol

This is mostly for myself. :drums:

It sounds like quite the set-up. All I have is a 500GB hard drive on my laptop with Plex. I'm sure one day it just won't cut it anymore.

Maybe it will always be good. Building this server is as much for my enjoyment of building it, as it is having it. Its like we like wrenching on cars (at least when we are building some thing cool)

I don't expect this to be cost effective. Certainly I could have just bought a 4K bluray player, and 4K TV for what is sitting in the chassis. But I like not having to swap discs.

I like the fact, that if I were so inclined I could watch any movee or TV show I own anywhere in the world.
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
So do you only have extremely high quality versions of your movies and shows, or do you just have an insane quantity of content on your server? Or both?

I rip straight Bluray and 4K Bluray's with no transcoding. The Plex server itself transcodes the raw MKVs to match whatever end client is playing them. If your smart TV has a Plex app, all the content will play on it. Your phones, iOS or Android, it doesn't matter. So each file is anywhere from 5-9GB for a DVD, 15-40GB for Blurays, and 60 to 120GB for 4K Blurays.

It will send the Raw MKV to my Gaming HTPC running Plex Media Player in Windows 10. The raw 4K HDR content with Dolby Atmos passes thru to my Receiver. its literally like plugging the disc in, no loss in quality at all.

I can also sync the movies to my phone directly, so if were to go on a road trip, I could have local copies on the phone at lower quality levels without having access to WiFi or Cellular data.

The sever is capable of 9x 1080P streams. And so far no troubles with 4K content in terms of Horsepower. But I've only tried 1 4k stream.



It will probably only be able to handle 2 or 3 4K streams at most.

Right now I have Atomic Blonde, Deadpool, John Wick 2, and Cars 3 ripped in 4K.

John Wick 2 is 77GB.

Title information
Name: John Wick: Chapter 2
Comment: FPL_MainFeature
Source file name: 00000.mpls
Duration: 2:02:23
Chapters count: 17
Size: 77.1 GB
Segment count: 1
Segment map: 0
File name: John_Wick_Chapter_2_t10.mkv

Some Plex pics.

26278816975_17ff7b12f9_o.png
[/url]

26212554151_8002b43b89_o.png


26006073590_ba99f1f751_o.png


26186513822_9fa3c8958a_o.png


]
32201692150_79120c7072_o.jpg


32201692570_ab500b16e2_o.jpg


34301203970_7b88b79604_o.png
 

Fish

From the quiet street
TCG Premium
Aug 3, 2007
40,518
7,871
Hanover Park
Real Name
Fish
Damn, that's some high resolution rips. Most of mine are just 4-6GB for the Bluray discs.

Same. I also dont have a 4K TV so I have not started 4K rips yet.

My 3TB drives are starting to get full though, so I do have to think of something shortly or just take stuff off that I dont want. Like stupid X files. :rofl:
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
What are you using for ripping 4K movies? Are you using MakeMKV with the updated Hash Files? That works pretty well?

I've got a 3DBluRay Rental subscription which lets me rent 4K discs just no way to rip them yet.
As long as the key is available yeah. Some of my 4k's don't have key's yet.

I did discover an issue with 4K HDR transcoding in Plex.

When Plex transcodes HDR content the 10bit BT2020c color space goes out the window and the transcode looks like a Zach Snyder film with crap muted colors. Plex hasn't offered a solution yet, and it turns out its been a problem for a few months. So we'll have to wait and see what Plex does, otherwise keeping 4K HDR content means you have to directly play it, no transcoding allowed. And that means no remote streaming because not many people have a Upload one 100Mbits or more. John Wick's bit-rate is 80Mbit/s on average, spiking over 100 when playing.

So for now, I'm storing 4K rips in a separate 4K library folder, AND keeping the original 1080p Bluray files around. So, John Wick 2 sucks up about 110GB of storage on its own.
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
Same. I also dont have a 4K TV so I have not started 4K rips yet.

My 3TB drives are starting to get full though, so I do have to think of something shortly or just take stuff off that I dont want. Like stupid X files. :rofl:

I dread how long it would take to Handbrake a 4K HDR title. My settings for H265 with regular 1080p stuff make for only 4-7FPS. 4K would be 25% that speed.

Hence the storage requirements. It takes about 45 minutes to 1hr to rip a 4k disc anyways.
 

muskie

TCG Elite Member
May 26, 2011
1,749
317
Palatine
As long as the key is available yeah. Some of my 4k's don't have key's yet.

I did discover an issue with 4K HDR transcoding in Plex.

When Plex transcodes HDR content the 10bit BT2020c color space goes out the window and the transcode looks like a Zach Snyder film with crap muted colors. Plex hasn't offered a solution yet, and it turns out its been a problem for a few months. So we'll have to wait and see what Plex does, otherwise keeping 4K HDR content means you have to directly play it, no transcoding allowed. And that means no remote streaming because not many people have a Upload one 100Mbits or more. John Wick's bit-rate is 80Mbit/s on average, spiking over 100 when playing.

So for now, I'm storing 4K rips in a separate 4K library folder, AND keeping the original 1080p Bluray files around. So, John Wick 2 sucks up about 110GB of storage on its own.

I've had the issues you describe with HDR as well. Plex won't direct play an MKV with HEVC at least on the Xbox One S. The Nvidia shield should be able to do any format.

I've just been downloading the remuxes as I don't have a BDXL reader, but I just downloaded the same John Wick 2 file as you and it was 89GB.

As of right now I've just got a little portable SSD that I've been copying the files to when we wanna watch HDR 4K and plug directly into the TV. Not ideal but it works for now.

I've been having some luck getting the video to direct play on the Xbox One if I remux into an mp4 with ffmpeg. I still have to transcode the audio but a potato could do that. The video stream direct plays, granted without 10 bit color, but it still looks 'Ok'

I'm with you though I want full quality rips. I just picked up a Lenovo SA120 12 disk DAS so I can expand my pool. Need to save up for some 8TB drives. Just don't want to think about what a resliver is going to take on a RAIDZ2 with 8TB drives.
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
Also, Xbox One app still doesnt seem to pass HDR through. It never pops up saying an HDR video is playing. What GPU do you have in your HTPC that is doing HDR for you?
\

The Gaming HTPC can play it back via software alone, it has a i7-4790k in it. I use it for Handbrake encodes from time to time. The GPU is a Asus STRIX Nvidia GTX980 4GB. The machine runs Windows 10, and boots straight into Steam Big Picture mode with an IR remote, and a Xbox 360 controller receiver so I can use my old XB360 controllers. I have Plex as a "game" link and I can just select it instead of a game and load Plex Media Server to watch movies.

Because its a full blown computer, it supports basically everything and will bitstream all the HD Codecs straight to the Receiver for decoding. So 4K HDR with Dolby Atmos support is no problem.

The thing about Plex is that it relies on the Client device's capabilities and the server transcodes to the client format. So If MS hasn't installed the codecs required to decode 4K HDR or say Dolby Atmos, then Plex will automatically transcode to the clients native format based on a Xbox 360/One/One S profile.

For an Apple TV thats usually MP4 container with H264 codec, and not sure what Audio streams it supports.

Is this the Plex you're referring to?

https://www.plex.tv/

Yeah, don't need a NAS to run it. You can just install it as a service on a Windows, Mac, or Linux machine and point it to a folder.

Movies and TV Shows in separate folders.

As long as its named correctly Plex will go out to TheMovieDB.com, or IMDB's website and scrape all of the meta data for you. Name, artwork, synopsis, cast listings. And if you have the Plex Pass it will also go out and get the special features for you too like Behind the Scenes featurettes, or deleted scenes....

Example:
Code:
/Media/Movies/Movie Name (Year).ext
/Media/Movies/Black Panther (2018).mp4


/Media/Movies/TV Shows/Show Name/Show Name.S1E01.Episode Title.m4v
/Media/Movies/TV Shows/Star Trek - Discovery/Star Trek - Discovery.S1E01.Pilot.mov
 

Thirdgen89GTA

Aka "That Focus RS Guy"
TCG Premium
Sep 19, 2010
19,299
15,654
Rockford
Real Name
Bill
Someone asked what it was liking using Plex when you aren't at home?

So, while at work I tethered my laptop to my phone and recorded a short clip. Just randomly clicking around and playing brief bits of Music, TV, and Movies, along with playing some 4k stuff so I could show how decently it transcodes.

https://www.youtube.com/watch?v=7EoTVjJ1_us&feature=youtu.be
 
Old Thread: Hello . There have been no replies in this thread for 90 days.
Content in this thread may no longer be relevant. Consider starting a new thread to get fresh replies.

Thread Info