My Anti-Skate: That time when I wiped out all the Finance data

A minuscule typo, poor controls, and a big fat oops.

a red headed marketing person making the walk of shame to the IT department

Fellow Substacker, Frederic Poag, posted a banger of a piece a while back about the “skate”. In it, he patiently tells a tale of someone who fucks up, and instead of owning their mistake and accepting the consequences, they hide, obfuscate, and (attempt) to dodge the ramifications of their fuckup.

It is a paean to the Trump voters who are now claiming that they didn’t know what their vote would lead to.

The post is here:

Trump Voters Knew What They Were Voting For

If you haven’t read it, I highly recommend it, even if it doesn’t feel like it is getting to the point (because Frederic masterfully sets the table).

In the comments I made a glib response of one time I could have skated and gotten away with it, but instead, I owned up to it, and the shitshow was avoided.

This is that tale.

Circa 2005, I was working at a small-ish company. Our business unit made measurement and test gear for industrial applications, and I was a Product Manager/Marketing person.

Our building was a shitty old IBM call center a stone’s throw from the Tucson International Airport. If you know anything about IBM from the 1980’s and 90’s, they were (and still are) a tech behemoth, but they were users of an archaic networking that wasn’t ethernet. They invented this homeless abortion called “Token Ring” and it was just hot garbage.

Needless to say, the building didn't have Cat5 cabling, it had this shitty Token Ring coax.

So, in the office, our network was WiFi and at the time that was limited to 10 mbps speeds. Like really fucking slow.

If you needed more bandwidth, you had to go to one of the conference rooms that had an honest to god fast-ethernet connection. That is 100 mbps, and if you needed to push any real data around, you did your work in one of these conference rooms.

One day, I spent an afternoon restructuring and rebuilding the marketing data repository on our main file server. We had one, and I basically built a new directory structure that was clean and tidy, also more navigable, and intuitive.

Pretty slick if I do say so myself.

Once I had it the way I wanted, I went to go remove the old structure, as we didn’t have unlimited storage.

And here’s where it gets bad.

I was (and am) a Mac user. I am comfortable diving into the terminal and using the command line. I have been for a long time.

And we were a medium sized business, without a very sophisticated IT group. We had two techs on site, a bigger group in our headquarters. But there was almost no security, or Access Controls, RBAC¹, or other policies to segregate user access and rights.

(If you are an IT person, this should be sending HUGE red flags to you).

So, to remove the old marketing directory, I typed the standard Unix command rm -rf ./*

Or, so I thought. I missed the initial period, and if you know unix at all, you know that I just told it to wipe out everything on that volume of disk.

It took me about 10 seconds to realize that I was wiping the server out. Since I was connected by cable, and a fast connection (for that era) the data was being deleted post haste.

I grabbed that cable and unplugged it, halting the command from recursively deleting every file and directory.

The damage was done, about 2 gigabytes of Accounting data (finance) was deleted.

I faced a quandary. I could just put my head down and pretend it didn't happen (and I know for a fact that we didn’t do good logging of activity, we were that unsophisticated) and likely get away with it.

Or, I could hang my head in shame and walk down to the IT team area and fess up.

I chose the latter, and their response was not what I expected.

I anticipated getting chewed out, having my wings clipped, and being made the butt of jokes in the office.

Instead, they laughed, and told me that it was good that I came to them, because they did a nightly incremental tape back up of that server, and in about 45 minutes they had restored that data and no one was the wiser.

I learned two lessons:

1. Never fuck around with SU mode, and to triple check when I am issuing a command to do something in a batch like that

2. Stand up and take ownership. Sure, I could have been yelled at, and perhaps disciplined, but it would have been ten times worse had I tried to hide the goof.

The company not long after that began to actually invest in IT systems, instituting Active Directory, and access controls to prevent people from gazing at data they shouldn’t (in reality, I should NEVER have had access to even see the accounting data, let alone the ability to modify/delete that data).

The coverup is always worse than the crime/activity.

---

1. RBAC is role based access control - pretty standard stuff now, less so in 2005