Puget Systems print logo
Read this article at https://www.pugetsystems.com/guides/944
Brett Nordquist (Customer Experience Engineer)

Buried by Storage Options

Written on April 28, 2017 by Brett Nordquist

It's 10 PM on a school night, and I know where my kids are, but not my files. 

It all started when the post office delivered a manila envelope, sealed with enough packing tape to wrap a small country. That was the first clue that my father was the sender. 

I found another envelope inside. I opened that to find a flash drive full of pictures he'd taken over the past decade or two. I finally remembered that I'd offered to create a picture story for their upcoming 50th wedding anniversary. 

So I began the process of copying thousands of pictures from the flash drive to my computer. But where should I save them?

Do you recall a time when your computer housed a single hard drive? I would spend hours meticulously organizing my folders and files. Windows Explorer was too pedestrian for my tastes so I experimented with a number of Explorer alternatives such as xplorer2 to organize all my documents and budding MP3 collection. 

For as much pride as I used to take in organizing my files, I couldn't have moved further from that today. 

I now have more local storage than I know what to do with. I boot Windows off a fast SSD, and store my music, video and documents on two platter drives. And this is where setup runs into trouble. I'm never quite certain what applications should run off my SSD and which should remain on my platter drives. And should I accept their default save locations or change them based on my configuration? 

Of course, I run all my games and Photoshop off my SSD, but what about Office or Turbo Tax or Dropbox? Or iTunes. Oh man, iTunes. I'm scared to look at how many iTunes library folders I have saved over the years. This is what keeps me from updating my PC. There's no way I could import the latest iTunes library to a new drive if my life depended on it. 

And speaking of Dropbox, cloud storage has only complicated my situation. I began saving files I wanted to share on an old Microsoft application called Mesh. Dang, I wish Microsoft would have left that little program alone. I then began using DropBox which I still use today. But when I installed Office 365, OneDrive came alone for the ride, and applications such as PowerPoint aggressively push saving files to it. 

That brings me to my lonesome Western Digital external drive. It's tiny. It's USB. And from its name (MyPassport) it sounds like it's going places. In reality, it sits in my desk drawer, and only sees the light of day when I'm shamed into backing up files to it after reading articles like this one from Mr. George. 

The reality is that I have too many storage options at my fingertips, and eventually it's going to bite me in the butt. 

Each year around the end of March, I play a game called, "Find the final version of last year's tax form." A decade ago, the game lasted 15 second. Last month I found no fewer than five 2015 tax documents scattered across my local drives, cloud services and external drives. Finding them was the easy part. Determining which was the final draft took a lot longer. 

This may be a pipe dream, but what I want from storage is similar to what I have with my iPhone. I don't want to think about where I save filed. I want a powerful search across all my storage devices/services. And I'm willing to pay to make sure everything is properly backed up. Basically, I want Windows to take the wheel and do all this for me. 

Today I have more storage options than ever before. But I'm not sure I'm any better off than when I had one 10 GB drive in my PC and drawer full of Zip drives and that one Jaz drive I never quite trusted. 

My goal is to have a storage plan in place before my parents celebrate their 50th wedding anniversary next month. I know I'm flirting with disaster, and I'd rather not have to tell my spouse again how I lost the first four months of pictures of our second child. 

Wish me luck. 

Tags: Storage, Files, Organization

A Synology (or any good NAS really) has solved some of this for me.

Posted on 2017-04-29 02:58:59

I've got a friend who swears by his Synology. I'm not thrilled about having another device at my home to manage but it would solve a number of my problems. What model Synology do you use?

Posted on 2017-04-29 05:38:16

I have an older 8-bay DS1813+. It has been replaced twice with newer models, the DS1815+ and DS1817+. The DS1515+ and DS1517+ are 5-bay equivalents. What's nice about the Synology is that I could start out with a bunch of mismatched drives of different speeds, sizes, and vendors as either SHR1 or SHR2 (1 or 2 drive redundancy), and that's largely why I went with an 8-bay unit. Over time I've replaced failed drives with larger ones, or the smallest drive when I needed more room to expand. I'm almost all 6TB WD Red drives now. As far as redundancy, generally SHR1 makes sense under 5 drives and SHR2 makes more sense when you get more than 5 or 6 drives. You used to have to format the array and start all over to switch between them (not very fun!), but now you can go from SHR1 to SHR2 later without losing data, however it's a one way trip. With SHR2, you'll have to replace more drives to see any increase of storage, due to the double redundancy.

Expect to get 1Gbps performance from a single computer. Even if you team 4x1Gbps from both Synology and a desktop, you'll still only get a single SMB stream from 1 computer, so it's great for backing up or having many machines in the building access it at once, but it won't be fast enough for live editing. I hacked my Synology to enable SMB 3.0 Multichannel on Samba 4.4.0 or newer, but it's not implemented well yet and didn't give me a great performance boost like I had hoped. I'm just saying all that to say that using a single 1Gbps connection keeps it simple and anything more complicated doesn't help too much with a single computer. The latest DS1817+ and DS1517+ models will support a 10Gbps card though, and that I expect will make a great deal of difference if you put a 10Gbps network card in your computer as well. Since I use mine for backup, it's not enough to compel me to upgrade for 10Gbps speeds, but I could just move my drives over to a new model without losing data if I ever wanted to.

One more remark: I run a media server called Plex on the Synology. It works well for streaming video, audio, etc. around the house. We have a Roku upstairs, an Apple TV downstairs, and couple iPads. I can pause a movie or song on one device, pick it up on another, and it remembers my spot. The Synology's CPU is not powerful enough for transcoding in real time, especially something like 4K video, DTS HD Master surround sound, or FLAC lossless audio format, but it can stream it direct if your device can handle it. You could also run Plex off a more powerful PC for on the fly transcoding, but then your PC would have to be on anytime you wanted to access your library. The Synology uses less power over a 24hr period compared to my desktop though when I monitored it with a wattmeter, as the drives spin down when it's not in use. So, as a backup and media server, we really like it!

Posted on 2017-04-29 12:13:32

Aaron, thank you for the great information. I'm going to take a look at one of the 5 drive models. I really like the idea of being able to use drives of different speeds and sizes, at least to begin. I'll have to check out Plex which you mentioned.

Posted on 2017-04-29 19:42:46

There is a RAID calculator on their website. Set it to SHR1 and drag your current drives down into the first 5 bays to see how much space you'll get with mixed drive sizes. There will probably be some waste, but it's more efficient than RAID 5.

Posted on 2017-04-29 22:35:03

I should also note: it will format the drives and you'll lose the data on them if you use existing drives!

Posted on 2017-04-29 22:36:32

"Finding them was the easy party."
That's one easy party.

Posted on 2017-04-29 04:56:54

Oops! I did throw myself an "easy" party after finishing my taxes this year. :-)

Posted on 2017-04-29 19:43:26

"I'm scared to look at how many iTunes library folders I have saved over the years. This is what keeps me from updating my PC. There's no way I could import the latest iTunes library to a new drive if my life depended on it."

I've got a similar issue with my wife's computer and her games. She's got hundreds of them. One of her friends has thousands. Really. Thousands of games (I just looked it up: 2,916 games on Steam, alone). There's no standard for where they store their configuration or save files (My Games would be too hard for them, I guess). I can restore an image of the drive from monthly backups (plus whatever Windows' File History thinks it's doing (hah)). But, as for putting a new copy of the OS on there and starting clean, I guess it'll never happen.

Also, another person mentioned a NAS. I have one and it's a good idea. But, it just leads to more of the same problem with where things are stored. Except for My Documents, I've changed Window's special folders so they all point to locations on the NAS. Every year, I move or copy most of My Documents (and some unique folders like my Calibre library) over to the NAS into a yearly folder under Archives. So, where's that document? It depends on how old it is and what kind it is.

Another problem is the stuff from the phones (photos, in general). The data is physically on them and I back them up to Google Drive. But, I don't archive them to the NAS. Plus, there's the issue of access to the wife's Drive account (I need to think about that, since she's certainly not going to worry about backing anything up). So, where's that picture? Depends on who took it, when, and with what.

About the only thing I'm sure of regarding the location of my data is: Microsoft, Google, and the NSA have it.

Posted on 2017-04-29 14:29:03

Haha, maybe the NSA needs to allow us access to our individual archives. Goodness knows we pay for it! :-P

Posted on 2017-04-29 14:57:32

You bring up a really good point with the games in Steam. I hadn't thought about that, but I've got a few dozen games there plus some in the Origin client. I honestly have no idea where those are stored.

Good call on the NSA.

Posted on 2017-04-29 19:44:58

I have two synology NAS systems now, my first one doesn't have enough storage, and then where do I backup a few terabytes of data. So I bought a four drive version, put drives in it and it sat for awhile in storage. When I finally went to use it the drives that are compatible with the Synology NAS have double/tripled in size, so it really isn't living up to it's potential. I guess I need to buy new drives for both of them. How do you temporarily move around terabytes? Very slowly and carefully, it is not easy. I've also got a drawer full of old hard drives to go through and even some floppies. You know how now when a parent dies, the children have to go through boxes of photos and papers to pass around to their siblings? What will they do when I die? They will have petabytes of data to look through, tons of photos on facebook, now drone footage and gopro movies. There is a service someone can start, clean up your deceased families digital footprint.

Posted on 2017-04-30 18:04:20

This is why I'm concerned about adding another device at home that requires any babysitting. I wish I could push everything to the cloud and access it from any device. I'm sure I could that with Amazon. But I need to organize it first or I'd just drown in so much data.

Posted on 2017-04-30 19:10:18

That is also my plan, with unlimited storage on Amazon and a plugin for Synology that will backup to your Amazon account it is a good combo. But I have the same issue, getting documents especially, organized first. My Fujitsu scansnap just added a scan to cloud feature, automatically naming documents via contents and choosing location by type. So if I just get the folder structure decided on and the workflow (what to save after scanning, what to shred, how to save some things for a certain time, etc), then it could be pretty easy!

Posted on 2017-04-30 19:24:52