IRS: So… Our Computer Crashed And Erased All Of Lois Lerner’s Emails

Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you:

No adverts like this in the forums anymore.
Times and dates in your local timezone.
Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance.
Email notifications for threads you want to watch closely.
Being a part of the oldest wargaming community on the net.

If you are already a member then feel free to login now.

**whembly**

streamdragon wrote:
whembly wrote:
If it's anywhere close to industry standards... then, they're likely lying through their teeth.
Which we have covered repeatedly saying "No, the federal government isn't anywhere close to industry standards".

Right... but, I have a hard time believing that.

I work in the IT industry and I'm extremely curious how it's all architected. Having that information, that'll provide insight as to whether any restoration is possible.

**motyak**

whembly wrote:
streamdragon wrote:
whembly wrote:
If it's anywhere close to industry standards... then, they're likely lying through their teeth.
Which we have covered repeatedly saying "No, the federal government isn't anywhere close to industry standards".

Right... but, I have a hard time believing that.

You don't say.

**dogma**

Jihadin wrote:

The Federal Judge gave a verbal order and its on paper. They have to comply with a Court Order. No Subpoena required.

To nondescript parties that may no longer work at the IRS, assuming they ever did. Absent an explicit list of names, which would basically be a series of subpoenas, these people have to do absolutely nothing.

Hell, the court doesn't presently seem to know who the relevant people are. Indeed, they may not even know who they are as they may not have been aware Lerner's drive was the one they were working on.

**streamdragon**

whembly wrote:
streamdragon wrote:
whembly wrote:
If it's anywhere close to industry standards... then, they're likely lying through their teeth.
Which we have covered repeatedly saying "No, the federal government isn't anywhere close to industry standards".

Right... but, I have a hard time believing that.

I work in the IT industry and I'm extremely curious how it's all architected. Having that information, that'll provide insight as to whether any restoration is possible.

Two words should showcase how the architecture is implemented:

Lowest
Bidder

Keep that in mind whenever you think about trying to compare the private sector to the public sector.

**Grey Templar**

You know the Private sector tends to gravitate towards the lowest bidder too.

**Jihadin**

dogma wrote:
Jihadin wrote:

The Federal Judge gave a verbal order and its on paper. They have to comply with a Court Order. No Subpoena required.

To nondescript parties that may no longer work at the IRS, assuming they ever did. Absent an explicit list of names, which would basically be a series of subpoenas, these people have to do absolutely nothing.

Hell, the court doesn't presently seem to know who the relevant people are. Indeed, they may not even know who they are as they may not have been aware Lerner's drive was the one they were working on.

Sullivan did not target any parties. He targeted IRS as a whole concerning the lost info

Judge Emmet Sullivan gave the government 30 days, until Aug. 10, to file a declaration signed by “appropriate” IRS officials under oath — sworn to be the truth under penalty of perjury — addressing the lost emails. The judge said the declaration, which should also include ways to recover the emails, will help inform the court about whether there is a need for limited discovery as requested by conservative watchdog Judicial Watch.

Also, we are forgetting

The judge also assigned federal magistrate John Facciola, an expert in e-discovery, to find out if there is another way to retrieve the emails.

**whembly**

streamdragon wrote:
whembly wrote:
streamdragon wrote:
whembly wrote:
If it's anywhere close to industry standards... then, they're likely lying through their teeth.
Which we have covered repeatedly saying "No, the federal government isn't anywhere close to industry standards".

Right... but, I have a hard time believing that.

I work in the IT industry and I'm extremely curious how it's all architected. Having that information, that'll provide insight as to whether any restoration is possible.

Two words should showcase how the architecture is implemented:

Lowest
Bidder

Keep that in mind whenever you think about trying to compare the private sector to the public sector.

Uh... sure, that's how the real works too man.

**dogma**

Jihadin wrote:

Sullivan did not target any parties. He targeted IRS as a whole concerning the lost info

I know, but our conversation is about the Walton ruling, not the Sullivan ruling. The Walton ruling pertains to the people handling the drive itself.

whembly wrote:

Uh... sure, that's how the real works too man.

Only in the case of particularly large corporations where the rules of bureaucracy come into play and the pool of bids for a contract is relatively large.

**whembly**

So... on the IT aspect of this scandal...

What this guy said:
Is the IRS Scandal Microsoft's Fault

Hmm….maybe Microsoft MSFT +1.52% is to blame for this whole IRS-email thing.

Just last week, “…a second federal judge has now ordered the IRS to explain under oath how the agency lost emails from former division director Lois Lerner, the woman at the heart of the Tea Party targeting scandal. U.S. District Court Judge Reggie Walton told Obama administration lawyers on Friday he wants to see an affidavit explaining what happened with Lerner’s hard drive. The IRS claims her computer suffered a crash in 2011 that wiped her email records at the time clean.”

Ah-hah! It’s a brilliant ploy. Can’t provide emails requested by the courts? Then just blame the computer guys! The computers crashed. It was a blue screen of death. You know what’s it like with Windows, right? We’ve all had this happen to us before. Who hasn’t had their computer crash? Curse you, Microsoft! This must be your fault! You’ve foiled us again!

I don’t know if the IRS did anything wrong – that’s for the courts to decide. But I do know a little something about information technology. And this is an IT issue. Judges. Lawyers. IRS people. Congressmen. The Media. Everyone’s trying to get to the bottom of the lost emails. But there’s one group of people missing from the conversation: the tech people. I don’t mean the ones who work for the IRS. They’re scrambling, I’m sure. The ones who should be subpoenaed are the ones who work at IT firms, like mine, across the country. They will tell a different story. When we hear that emails were just “lost,” especially in 2011, we scratch our heads in amazement. Ask any IT firm, the clients they serve, or the IT people that work in corporate America: e-mails don’t get “lost.” And frankly, computers don’t really crash very often. That was a great excuse in…oh…1995. But not anymore.

In other words: you can’t blame Microsoft. Nice try.

Let’s assume that Ms. Lerner was using Microsoft Outlook as her email client. Outlook is still the most popular email software around (at least for now). Outlook is a client-side software which means that it’s installed on a desktop computer. It does save data. But it’s reliant on a server-side, email messaging platform in order to send out emails, like Microsoft Exchange or something similar. Sometimes a copy of that email does reside locally. But every IT firm I know who sets up products like Exchange (particularly in these times of cheap storage space) also configures it to save emails on the server as well. This enables the IT staff to administer users’ mailboxes, security, rules and, of course, backup the data. And it enables users to access their email online or from any other device they choose. In today’s corporate environments (and that includes 2011) it’s actually more difficult to setup a server-based email system where emails are NOT centrally saved.

In today’s corporate environments, computers running even older versions of Windows are more portals than desktops. As explained above, emails are almost always by default saved on a central server. Ms. Lerner likely did her document and spreadsheet work (assuming that’s the kind of thing a director actually does) on files that were automatically stored on a server somewhere else too. Even if she was saving this work locally, it’s Network Administration 101 to have these files automatically backed up to another location. Small businesses like mine pay around $25 per month to have all of our files automatically backed up using services like Carbonite or Mozy. I find it hard to believe that backups weren’t taken or stored since 2011 by the IRS’ IT team.

The “computer crashed” story also seems implausible for another reason: computers don’t really “crash” any more. OK, I admit that this may happen once in a while. But I just don’t see this happening at clients like it did many years ago. One reason is that so many of my clients are now using Apple AAPL +1.05% products and Apple products are really well made. But, say what you want about Microsoft, they learn and adapt. And Windows has become a very, very reliable system. Maybe Ms. Lerner’s version of Windows was a very old one but I would have to go back pretty far before Windows XP to find a version of Windows that would crash a hard drive so bad that it couldn’t be recovered. Was it a virus that did this? Something that snuck its way into the IRS’ network? That’s more of a concern to me. Not Windows.

And one other thing: Every accountant (like me) knows that the IRS’ record retention policy ranges from 3-7 years for businesses. We generally use seven years as a rule of thumb for our clients. Imagine me telling an IRS auditor that I couldn’t retrieve the information requested from two years ago (this investigation began in 2013) because my “computer crashed.” Even my smallest clients back things up and have them stored for years – especially server based data stored in their accounting and…yes…email systems. It’s too easy and inexpensive not to do this with today’s services.

So did Ms. Lerner’s computer really crash, wiping out all her data, including emails? That story’s really hard to believe for anyone with an IT background. It’s easy to blame Microsoft for our problems. That used to be a great excuse. But that excuse is getting harder and harder for anyone, even the IRS, to make.

Besides Forbes, Gene Marks writes daily for The New York Times and weekly for Inc. Magazine.

Indeed Mr. Marks... Indeed.

**Dreadclaw69**

So the excuse has gone from "a dog ate my homework" to "a big boy did it"

**dogma**

U.S. District Court Judge Reggie Walton told Obama administration lawyers on Friday he wants to see an affidavit explaining what happened with Lerner’s hard drive. The IRS claims her computer suffered a crash in 2011 that wiped her email records at the time clean.”

Also, here is something that is completely contradictory.

Ah-hah! It’s a brilliant ploy. Can’t provide emails requested by the courts? Then just blame the computer guys!

How is this being done? The argument being made is that the relevant data from Lerner's drive could not be recovered; that is not a necessary indictment of the "tech guys".

**whembly**

dogma wrote:

Ah-hah! It’s a brilliant ploy. Can’t provide emails requested by the courts? Then just blame the computer guys!

How is this being done? The argument being made is that the relevant data from Lerner's drive could not be recovered; that is not a necessary indictment of the "tech guys".

By standing by "the dog ate my hard drive" story, yeah... I'd say it is an indictment of the "tech guys".

Judge Walton's order would hopefully give these "tech guys" a forum to explain what truly happened.

**dogma**

whembly wrote:

By standing by "the dog ate my hard drive" story, yeah... I'd say it is an indictment of the "tech guys".

How? The fact that data cannot be recovered from a drive according to SOP is not an uncommon occurrence, especially when dealing with drives used by people that don't understand technology (Lerner). So I really don't see how sticking with that story indicts the "tech guys".

whembly wrote:

Judge Walton's order would hopefully give these "tech guys" a forum to explain what truly happened.

Or hang themselves regarding their careers, assuming they can even be identified or subjected to the order. I mean, if I were in their position I would make myself as small as possible in the course of legal and media proceedings because I would gain absolutely nothing from being significant, and stand to lose a lot.

**Seaward**

Subpoenas are the most common cause of hard drive failure.

**Jihadin**

Seaward wrote:
Subpoenas are the most common cause of hard drive failure.

Jebus. Even the word subpoena make a hard drive crash. One heck of a verbal virus

**whembly**

dogma wrote:
whembly wrote:

By standing by "the dog ate my hard drive" story, yeah... I'd say it is an indictment of the "tech guys".

How? The fact that data cannot be recovered from a drive according to SOP is not an uncommon occurrence, especially when dealing with drives used by people that don't understand technology (Lerner). So I really don't see how sticking with that story indicts the "tech guys".

You can forensically recover a crashed hard drive. "Crashed" is the operative word... the "IT geek" in me is asking how did it crash?

Look at this image:

You basically have multiple storage "platters" with mechanical parts. If any of the "mechanical parts" broke... then, the forensic operator can rebuild it.

If the IT department destory'ed her hard drive via what I call the mangler, ie:

Then, yeah... duh, it can't be recovered. However, the department will need to describe how/why it was chosen to be physically destroyed.

Neverthanless... we still need a full accounting how the IRS' email system is architected. It's troubling that we don't know this yet...

whembly wrote:

Judge Walton's order would hopefully give these "tech guys" a forum to explain what truly happened.

Or hang themselves regarding their careers, assuming they can even be identified or subjected to the order. I mean, if I were in their position I would make myself as small as possible in the course of legal and media proceedings because I would gain absolutely nothing from being significant, and stand to lose a lot.

True... but, Walton's order is required under oath. Those key folks will be identified.

**dogma**

whembly wrote:

You can forensically recover a crashed hard drive. "Crashed" is the operative word... the "IT geek" in me is asking how did it crash?

Well, no, you can't forensically recover all the data on a recycled drive; only some of it; "recycled" being the key term here.

whembly wrote:

You basically have multiple storage "platters" with mechanical parts. If any of the "mechanical parts" broke... then, the forensic operator can rebuild it.

I know what hard drives look like.

Your comment would only be relevant if the the intact drive, or some of its platters, still could be found and that relevant data could be recovered from them.

whembly wrote:

Then, yeah... duh, it can't be recovered. However, the department will need to describe how/why it was chosen to be physically destroyed.

You're assuming that an IRS department handled Lerner's drive, and that the employees that it handled still work at the IRS.

whembly wrote:

Neverthanless... we still need a full accounting how the IRS' email system is architected. It's troubling that we don't know this yet...

What would that accomplish?

whembly wrote:

True... but, Walton's order is required under oath. Those key folks will be identified.

And they will probably plead the 5th, if they are identified; meaning that their names will be forgotten.

**whembly**

dogma wrote:

whembly wrote:

Neverthanless... we still need a full accounting how the IRS' email system is architected. It's troubling that we don't know this yet...

What would that accomplish?

To confirm the validity of their response.

whembly wrote:

True... but, Walton's order is required under oath. Those key folks will be identified.

And they will probably plead the 5th, if they are identified; meaning that their names will be forgotten.

Did we forget Lerner's name?

**streamdragon**

whembly wrote:
You can forensically recover a crashed hard drive. "Crashed" is the operative word... the "IT geek" in me is asking how did it crash?

Look at this image:
>image snipped<
You basically have multiple storage "platters" with mechanical parts. If any of the "mechanical parts" broke... then, the forensic operator can rebuild it.

If the IT department destory'ed her hard drive via what I call the mangler, ie:
>image snipped<
Then, yeah... duh, it can't be recovered. However, the department will need to describe how/why it was chosen to be physically destroyed.

Neverthanless... we still need a full accounting how the IRS' email system is architected. It's troubling that we don't know this yet...

I was under the impression forensic groups already tried, and failed, to recover data from her crashed hard drive? I'm pretty sure that is stated numerous times throughout this thread.

As to the grinder, which I don't think we've heard either way about yet, I can tell you that it is SOP in the government to destroy physical media (including hard drives, CDs, DVDs and thumb drives, among others) when it can no longer be used, especially if it has at any point held PII information (which Lerner's most certainly would have). So while I can't say IF the drive was destroyed, I can say that it would be absolute SOP to do so. When I was at Anti-Trust, we had a giant shredder truck come around once a month. Loud as all hell when it hits the hard drives.

**whembly**

streamdragon wrote:
whembly wrote:
You can forensically recover a crashed hard drive. "Crashed" is the operative word... the "IT geek" in me is asking how did it crash?

Look at this image:
>image snipped<
You basically have multiple storage "platters" with mechanical parts. If any of the "mechanical parts" broke... then, the forensic operator can rebuild it.

If the IT department destory'ed her hard drive via what I call the mangler, ie:
>image snipped<
Then, yeah... duh, it can't be recovered. However, the department will need to describe how/why it was chosen to be physically destroyed.

Neverthanless... we still need a full accounting how the IRS' email system is architected. It's troubling that we don't know this yet...

I was under the impression forensic groups already tried, and failed, to recover data from her crashed hard drive? I'm pretty sure that is stated numerous times throughout this thread.

Nope... typically, from what I understand, the agency would request that resource from the FBI. (they're one of the best, because they get to have all the cool toys

)

As to the grinder, which I don't think we've heard either way about yet, I can tell you that it is SOP in the government to destroy physical media (including hard drives, CDs, DVDs and thumb drives, among others) when it can no longer be used, especially if it has at any point held PII information (which Lerner's most certainly would have). So while I can't say IF the drive was destroyed, I can say that it would be absolute SOP to do so. When I was at Anti-Trust, we had a giant shredder truck come around once a month. Loud as all hell when it hits the hard drives.

I'm pretty sure they said were destroyed, but the current head kinda walked back that statement.

I still don't think we definitively know... yet.

Also, keep in mind that the IRS was being sued by the Isreali non-profit group prior to this event. The SOP, should've been, that these things be saved in case it's needed for discovery.

**dogma**

whembly wrote:

To confirm the validity of their response.

And, should the response be confirmed as unsupportable, what do you think would happen?

whembly wrote:

Did we forget Lerner's name?

Lerner was a senior IRS official, not a guy that tried to recover a hard drive.

And, let's be honest, there aren't many people in significant hiring positions that would care enough to remember the name of IT guy X who was sort of involved in the IRS scandal.

**whembly**

dogma wrote:
whembly wrote:

To confirm the validity of their response.

And, should the response be confirmed as unsupportable, what do you think would happen?

That those who testified to Congress committed perjury.

whembly wrote:

Did we forget Lerner's name?

Lerner was a senior IRS official, not a guy that tried to recover a hard drive.

And, let's be honest, there aren't many people in significant hiring positions that would care enough to remember the name of IT guy X who was sort of involved in the IRS scandal.

If the IT manager/Email admin pleads the 5th, I guarantee you those names would be remembered.

**dogma**

whembly wrote:

That those who testified to Congress committed perjury.

That's not a thing which can't presently happen.

But, to entertain you, the absolute worst case scenario for anyone who stood before Congress is for evidence to be produced that supports a perjury case. This does not mean they will be tried for perjury.

whembly wrote:

If the IT manager/Email admin pleads the 5th, I guarantee you those names would be remembered.

Sure, by people like you, but most people would forget it within the year; much more quickly than the name would be forgotten if a person with either title provided testimony.

**cuda1179**

I was reading about this on another site. Basically the manufacturer stated that the chances of a hard drive failing in the manner described was 1 in 36 in any given month. The chances that all 7 people in question having their computers fail in the timeframe stated by the defense mean that someone is 340 time more likely to win the lottery, with the powerball number, plus the megamillions number.

**Jihadin**

cuda1179 wrote:
I was reading about this on another site. Basically the manufacturer stated that the chances of a hard drive failing in the manner described was 1 in 36 in any given month. The chances that all 7 people in question having their computers fail in the timeframe stated by the defense mean that someone is 340 time more likely to win the lottery, with the powerball number, plus the megamillions number.

Dammit....some people have all the luck.....wait...

**Ouze**

cuda1179 wrote:
I was reading about this on another site. Basically the manufacturer stated that the chances of a hard drive failing in the manner described was 1 in 36 in any given month. The chances that all 7 people in question having their computers fail in the timeframe stated by the defense mean that someone is 340 time more likely to win the lottery, with the powerball number, plus the megamillions number.

This is a fascinating example of a lie of omission made by someone technically knowledgeable enough to know that the layman won't have the ability to know this isn't true. Forget for the moment we don't know which manufacturer provided the drives in question, and that the exact rate of failure is a trade secret that that this guy is only 33% likely to know - we don't need to know that.

The chance of any individual drive failing is relatively low regardless of manufacturer, yes. However, that's burying the fact that manufacturers often have bad runs of drives - for example, Seagate recently had a series of drives with a bad firmware that led to failures very early in the bathtub curve unless you flashed the firmware (coincidentally, they were manufactured just before the window in which the alleged crashes occur; though no one knows if the drives in question are Seagates). If you're a very large vendor who buys a great deal of hard drives in bulk, statistically you will have a great deal of hard drives that are all part of a bad batch.

For highest amounts of data security, hard drives can be put together in something called a RAID array. This is a set of drives, 2 or more, that have the data either mirrored or divided up among them, The exact functionality is beyond the scope of this post, but the idea is that if a single hard drive fails, you won't lose data. However, if you have a multiple simultaneous drive failure (2 drives if raid5), you lose it all. As such, most manufacturers of these systems recommend not using drives all made by the same vendor (i.e don't use 5 identical Seagate drives) to avoid having multiple simultaneous drive failures.

Google did a giant survey in 2007 that looked at drive failures in a population of, if I recall, like 25,000 drives and found that the biggest predictor of drive failure is manufacturer, because some just make really lousy hard drives. Unfortunately, they didn't name any specific vendor on this; but they found rates as high as 9% in the drives they own for specific vendors as I recall. If you are a large organization that buys drives in bulk, such as the IRS (or my company) you will definitely see very large batches of bad drives pop up (as I have seen firsthand in my company).

TL;DR the "1 in 36 failure/lottery analogy" is speculation based on zero concrete info pulled out of someone's ass while ignoring data that person certainly knows in order to feed a pre-ordained viewpoint.

**streamdragon**

Ouze wrote:
If you are a large organization that buys drives in bulk, such as the IRS (or my company) you will definitely see very large batches of bad drives pop up (as I have seen firsthand in my company).

We lost about 30 or so power supplies all in the same 2 week period when I worked at ATR. The IT guys were getting a little ... tense when people kept calling.

**d-usa**

streamdragon wrote:
Ouze wrote:
If you are a large organization that buys drives in bulk, such as the IRS (or my company) you will definitely see very large batches of bad drives pop up (as I have seen firsthand in my company).

We lost about 30 or so power supplies all in the same 2 week period when I worked at ATR. The IT guys were getting a little ... tense when people kept calling.

But my brother is a sys-admin and he tells me that a power supply only has a 3% change of failure in the first 5 years so you would have been more likely to win the lottery than losing all those power supplies.

You just perjured yourself on Dakka my friend

**motyak**

d-usa wrote:
streamdragon wrote:
Ouze wrote:
If you are a large organization that buys drives in bulk, such as the IRS (or my company) you will definitely see very large batches of bad drives pop up (as I have seen firsthand in my company).

We lost about 30 or so power supplies all in the same 2 week period when I worked at ATR. The IT guys were getting a little ... tense when people kept calling.

But my brother is a sys-admin and he tells me that a power supply only has a 3% change of failure in the first 5 years so you would have been more likely to win the lottery than losing all those power supplies.

You just perjured yourself on Dakka my friend

Quick steam, plead the 5th!

**d-usa**

motyak wrote:
d-usa wrote:
streamdragon wrote:
Ouze wrote:
If you are a large organization that buys drives in bulk, such as the IRS (or my company) you will definitely see very large batches of bad drives pop up (as I have seen firsthand in my company).

We lost about 30 or so power supplies all in the same 2 week period when I worked at ATR. The IT guys were getting a little ... tense when people kept calling.

But my brother is a sys-admin and he tells me that a power supply only has a 3% change of failure in the first 5 years so you would have been more likely to win the lottery than losing all those power supplies.

You just perjured yourself on Dakka my friend

Quick steam, plead the 5th!

If he does, nobody will ever forget the name "steamdragon"