I have a cupboard shelf full of big external hard drives going back to 2008, which also hold copies of my internal hard drives from my PC’s going back to 1998. Seems to work fine.
I was doing something similar, but manual processes are increasingly prone to error, and I’m more and more uncomfortable with a fractured dataset over time.
This is one of the topics of tonight’s show. I’m reworking my entire data management pipeline.
I just wrote a script that combed through all my hard drives, found all my iPhone photos, and put them in a single folder on my latest external hard drive. It’s taken a lot off my mind, as a lot of those are just for me. Other photos and videos are published on my blog or facebook or youtube and other places. But 90% of my photos are just for me and the sake of my memories.
I ordered it.
6x 8TB WD Red NAS drives (WD80EFZX)
1x Synology DS1517+
I’m going with RAID-5 (32TB usable space, one-drive redundancy). The sixth drive is a cold spare so I can immediately begin a rebuild of the array if a drive fails.
I’m going to back the whole thing up to Glacier with a semi-monthly sync.
Now that you’ve covered hardware backup solutions, maybe do a show on the software side of backups?
1 Like
Maybe once I get this glacier sync actually working.
Backup pipelines.
3 Likes
I’m considering btrfs (Wikipedia) the next time I replace/upgrade my linux raid—in particular, the raid I use for backups. I’m already using it for off-site backups.
Btrfs seems half awesome, half minefield.
Pros:
- Efficient snapshotting, including read-only snapshots—perfect for backups, my primary use case.
- Incremental snapshot transfer—for backup to an external disk, my secondary use case.
- Data duplication support in the style of raid1 without needing ~exactly matching disks that you’d otherwise want with raid1.
- More generally, device flexibility.
- Data and metadata checksumming for integrity checking, with an online scrubbing utility for integrity checking.
Cons:
- Basically still experimental, even though it was merged into the kernel in 2009.
- Less robust recovery tools if something goes wrong.
- Apparently easy to fuck up?
One of the amazing things about living in New York is that this was all same day shipping.
I’m actually setting it all up today! So far, the Synology configuration experience is night and day compared to my old Netgear duo.
1 Like
We use btrfs here at work, I don’t manage/build the servers here so I have no choice in the matter, and it’s worked really well for the last couple years. I wish I could provide better details other than “been working fine here!”
1 Like
The Synology uses btrs by default unless you specifically tell it to do otherwise.
I tried to convert a server to btrfs from ext4 and found out the btrfs-convert tool is experimental and kind of broken.
No matter how trusted and well tested, you should always backup your data to another file system before attempting any kind of file system conversion. But wait, if you’re going to do that you may as well just have the new file system on the other drive, copy everything over, and be done with it.
In-place file system conversion is a tool for emergency situations only.
2 Likes
Or for cheapskates like me who didn’t want to pay for another disk just to convert the filesystem. I didn’t lose any data by the way.
Wow, you are way braver than I.
That is a horror story.
2 Likes
To be clear: you don’t think you lost any data.
If the new Synology NAS I got is a SR-71 Blackbird, my old ReadyNAS NV+ is the plank of wood the Red Baron flew around on.
Anyone want a plank of wood? It has 4x2TB drives in it, and a 3rd party power supply, and it’s very dusty, but it appears to still work just fine. You’ll just have to wait until I finish backing up, erasing, and resetting it. The most I would ask for the whole thing is $200 considering that to buy an empty one with brand new drives would be something close to $50+$85x4 that’s a maximum price of %50 off.
New 2TB drives are like $85. 85x4 = $340.
EDIT: Ok, I guess they aren’t. Let’s reset the NAS price to $120 then.
Bah, that’s not a NAS. This is a NAS:
Okay, I kid… Mostly because I used to work on these a few years back.
For personal/home use, I’ll need to think about that ReadyNAS price though… It is tempting.
I really just want it out of my house more than anything, but it does work and does have value, and I am planning to buy new furniture soon, so…