Fail of Your Day

We wasted several days of machine time at work because some guy had daily backups of his entire email account adding up to well over 1tb of pst files. We don’t even care about that account.

How could one person have 1TB of emails?

By literally making a copy of their PST every day. The software we use would eventually parse them all and dedupe it, but after 5 days of processing we killed it to see what the problem was. A normal desktop usually takes 4-24 hours to process so we usually just run them overnight and don’t pre-filter things.

That’s a pretty ridiculous architecture to begin with. Upload the entire backup and THEN dedupe it. It should be simply uploading a diff of all changes since the last backup. The end.

My laptop backs itself up every 20 minutes. If the backup software is doing a shitty job, that is so not my problem.

I have to say I am so impressed with this Synology NAS out of the box. If I had a company I would buy a very large/enterprise model from them without hesitation, and I would fully invest in its ecosystem as far as backups were concerned.

Right?

Everything just worked. I remember the BS I had to do to set up multiple users and permissions on my old Netgear Duo whatever. This thing has a real user/permission model and it just works.

5 days of dedupe processing for 1TB of data? Seems ridiculously slow to me.

From my reading it was five days of processing because they were sending 1TB/day over a wire to a server where it entered a queue to be diffed.

It’s forensic processing, it’s slow. Somewhere north of 10 million items in the database before I killed it. A typical desktop would be more like 3-6 mil and a lot of that won’t have meta data that needs parsing.

1 Like

Still seems pretty slow to me, but I work on data deduplication, so maybe my point of view is a bit biased. It may also depend on how slow the link is to said server, and I admit I didn’t think of that.

Hmm, by “forensic processing,” do you mean actually reading the data and making some sort of sense out of it other than simply looking for similar looking byte streams for dedup? If so, then yeah, I guess I can see how that may be slow compared to what I work on.

Yeah, it’s a program called FTK which is basically the industry standard. It parses through everything, breaks out metadata, indexes text, generates hashes, etc. It is largely a disc IO bottleneck but I can’t exactly justify buying a 2TB NVME drive yet. As is I use an NVME drive for everything but the disc image and standard drives or SSDs for those depending on size.

Gotcha. Yeah, this does look more intense than just hashing similar looking byte streams of data, which is the kind of dedup I work on.

So when I was younger I got sick a lot. My mom insisted I was making it up, despite the fact that after taking me to specialists they told her my adenoids were a problem. She of course ignored all of their cumulative professional wisdom and made sure I felt shitty whenever I got sick.

Turns out my adenoids have been trying to kill me all these years. After I finish this month-long course of antibiotics I need surgery to get them out.

Bonus: My doctor was horrified and awestruck when he saw these scans. He has no idea how I’m able to breathe at all.

2 Likes

Which bit is the adenoids? Is it the bulge to the left of the cursor?

I should’ve labeled it. Here’s a better pic. Left is me now with my hypertrophic adenoids circled in red and right is what it should look like.

1 Like

hot damn

Holy fucking shit.

Got linked on r/subreddit drama. Was being a bit of a drunken tit, but everyone else wasn’t golden either. Still not fun to wake up to. And a reminder that reddit is a shit hole.

You want to cop to a link?