Help The Folks Flooded Out In Looziana

August 16, 2016

PAy It Forward

TAR and Keywords and Proportionality: Oh My!!

August 10, 2016

I’ve waited a bit to write this post because I wanted to see what my colleagues were saying about the latest opinion from Judge Peck. In ED circles, a new ESI opinion from Judge Peck is more highly anticipated than the next Bruce Springsteen CD, except maybe in the Facciola household where The  Boss is revered just below … well, actually I’m not sure his status is below that of anything in the Facciola home except, of course, Mrs. Facciola.

Earlier this week Judge Peck opined in the case of Hyles v. New York City ( No. 10 Civ. 3119 (S.D.N.Y. Aug. 1, 2016) that proportionality trumped TAR. And he didn’t beat around the bush about it, stating in the very first paragraph of the order:

“The key issue is whether, at plaintiff Hyles’ request, the defendant City (i.e., the responding party) can be forced to use TAR (technology assisted review, aka predictive coding) when the City prefers to use keyword searching. The short answer is a decisive “NO.” “

His reasoning was, of course, that the absent an agreement of the parties as to a specific search protocol, the applicable standard is the Sedona Principle 6, which holds that

Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information. (The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing Electronic Document Production, Principle 6 ,

Well the Twitterverse exploded with comments about how Judge Peck declined to order the parties to use TAR this and Judge Peck pulls back on TAR enthusiasm that. In the spirit of the impending football season all I can say is “Come on man”

First off, we all know that Judge Peck has never ordered anyone to use TAR.  He’s entered orders in several cases where the parties agreed to use TAR, a factor that had not happened in the Hyles case.

And this lack of a fundamental understanding of the fact that in the legal profession (that’s profession, not industry) the word “order” can be either a noun or a verb.  I lay this lack of understanding squarely at the feet of the ever increasing assimilation of eDiscovery software and services companies by people who have no legal background. I’ve said it before and I won’t go off on that particular rant again here.

Rather I’d like to just point out one part of the proportionality debate that seems to be missing.  Judge Peck in Hyles mentions cooperation and speed of process, and refers to the Tax Court decision in Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue 143 T.C. 9, 2014 WL 4636526 at *3 (2014, which spoke to the same considerations.

But in his decision, J Peck notes on page 3 that “… in general, TAR is cheaper, more efficient and superior to keyword searching.”.   I think that if I say “not so fast” one more time in a column that I’m going to hear from Lee Corsos attorneys but I have to say that I don’t believe the issue of “cheaper” has been clearly established.  Even in Hyles, J Peck says at Fn 2 that “The Court acknowledges that some vendor pricing models charge more for TAR than for keywords. Usually any such extra cost is more than offset by cost savings in review time.”

I respectfully argue that there has been no empirical validation of that statement that I have seen.  Now it may very well be that vendors have filed briefs in matters that address that point or even presented substantiation for such a positon during the submission of attorney fee claims in cases that I have not seen. So what I’d really like to see is a case study that shows the efficiency based on price savings not time savings of TAR on a particular set of documents.

Not that time savings is irrelevant or should not even be the deciding factor. But it should be just that: a factor. One factor of several to be weighed in the process of which tool to use.

I note with interest that David Horrigan,  E-Discovery Counsel and Legal Content Director at kCura, in a blog post on another case this week, the 10th US Circuit Court of Appeals’ decision last week in Xiong v. Knight Trans (see ) mentioned that  “We’ve always been skeptical of attempts to use the 1985 Blair and Maron study to argue that keyword searches are only 20 percent accurate”.  I’ve also disagreed with the general proposition that TAR is always better than keyword searching ( see my post Reports of the Death of Keyword Search Are Greatly Exaggerated at ) and I think the point here is the same one that Judge Peck makes on page 5 of the Order in Hyles.

“ It is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching.  … While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used (7/18/16 Ltr. at 4-5), the standard is not perfection, or using the “best” tool (see 7/18/16 Ltr. at 4), but whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B). “

And as one consultant in our field (who wished to stay above the fray so I will not use his name) said to me recently:

“Choice of predictive coding, managed review (with or without validated search), or just validated search does NOT pre-determine success.  … So a bad protocol might lead to poor results in all three, and a good protocol might turn south in all three if calibration and QC is missing, or if it is improperly applied.  Ultimately it is only “better” if a reasonable production is made without substantial critical documents left on the cutting room floor.”

As I said several years ago in another column about another issue   ( ) :

“It’s the archer not the arrow”.


What Do GC’s Really Want in an eDiscovery Vendor?

July 15, 2016

From this weeks Advanced Discovery blog:

For those of you who haven’t seen it, earlier this week Matthew Verga, the VP of Marketing Content here at Advanced Discovery, posted the third part of an ongoing series discussing insourcing and outsourcing of eDiscovery functions.  This week’s post is called ‘Models, Benefits and Challenges of Outsourcing’ and can be found here.  Part 1 is available here andpart 2 is available here.

Matthew and I also did a webcast, just over a week ago, on the same subject.  Called “eDiscovery Insourcing vs. OutsourcingFiguring Out What’s Best for Your Organization,” a recording of the program can be viewed by registering here:

But meanwhile, I thought it would be interesting to get an unbiased view of the topic and see if our perspective matched up with comments from other observers:

Ari Kaplan is a leading independent legal industry analyst and is the principal researcher for a variety of highly respected benchmarking reports.  He is also widely in demand as a speaker for events in Australia, Canada, the United Kingdom, and throughout the U.S.   Ari is an inaugural Fastcase 50 honoree and a finalist for ILTA’s Thought Leader of the Year award and, like both Matthew and Tom, regularly competes in Ironman triathlons.

OK, Tom and Matthew don’t compete in triathlons.  I just wanted to see if you were paying attention.

What we DO have in common with Ari is monitoring the eDiscovery space for trends and performance statistics.  To that end, every year, Ari Kaplan Advisors produces a market research report called E-Discovery Unfiltered: A Survey of Current Trends and Candid Perspectives.  It is based on independent, anonymous interviews with 25-30 professionals responsible for eDiscovery decision-making, and in 2016, Ari personally spoke with 9 in-house lawyers, 9 in-house legal administrators, and 8 AmLaw 200 law firm partners about vendor management trends and recorded their direct impressions of 20 companies, including Advanced Discovery.

Of the 18 respondents from corporations: 5 were in financial services, including banking and insurance; 4 were in life sciences; 3 were in retail; 2 were in technology; 2 were in manufacturing; 1 was in energy; and, 1 was in entertainment.  Of those 18, 65% worked for companies with revenues that exceed $10 billion, and 69% worked for companies with over 10,000 employees.

Among other topics, the report addressed where in-house legal teams are investing, the issues that e-discovery leaders are most concerned about, and the qualities that matter most to them when working with a vendor.  You can read more, including how to order a copy of the report, at

But what I found most interesting is the chart below, which was reproduced on the AKA web site. It shows some trends that Matthew and I have been discussing, and that the ACC has also reported in their surveys, namely that pricing is not the most important factor to either GC’s or litigation support personnel at corporations.

What IS important to them are things like technical skills, project management, communication, security, and service.  Price is important but not as important as performance.  Just as Matthew and I have been pointing out.

So read Matthew’s postlisten to our webinar, and look at Ari’s site.  I think you will find them most informative, especially regarding the skills that GC’s want from their ED vendors.  Performance not prices.


Is eDiscovery Really Too Expensive?

July 8, 2016

From my post on this weeks Advanced Discovery blog :

Those of us who work in the eDiscovery profession (and yes it’s a profession, not an industry . . . that’s a personal foible that’s best left for another day’s discussion) are constantly asked why the process is so expensive. A while back, one attorney actually posted in a discussion thread:

Why is EDD and Data Processing So Expensive? Why can’t IT or my secretary do this? I know that we have all been faced with this so I am wondering if anyone can point me to a few good references, as I want to write a document that the average person can read and understand. This will help from having to explain this 300 times a year.

The answers have ranged from the comically brief: “It runs on magic, and magic is expensive.” and “Why should lawyers get to make all the money?” to the concise and erudite answer posted by George Socha:

” ‘Why is EDD and Data Processing So Expensive?’: Processing ESI can be expensive because consumers demand that the work be done very quickly and with a high degree of accuracy and reliability, in conditions where it may well be that at the outset no one knows the scope of the undertaking or the complexities and challenges to be encountered along the way. Prices vary greatly, at least in part because “processing” means many different things. Prices have dropped considerably; as a result, however there now is the danger of consumers paying such a low price that providers end up cutting too many corners, with a drop in quality that may not be acceptable.”

But really to me the query seems to beg question, “what is ediscovery”?  Ralph Losey took a shot at the issue several years ago on his blog, The e-Discovery Team ( ) but it seems to me that anyone who truly understands what e-discovery is would know that the answers to the question (leaving aside the obvious comment that 65-70% of the cost of any e-discovery project is the attorney review, so if e-discovery projects are too expensive, the fault lies with the lawyers’ fees) are:

  1. Because you need special training, and
  2.  It isn’t too expensive if done properly

As to “Why can’t IT or my secretary do this?” If your IT folks or your secretary has an acceptable level of expertise and experience and are provided an acceptable level of resources, then they CAN do the work.One is not able or unable to handle the challenges of dealing with ESI simply because of the name of one’s employer or the title of one’s position. An organization does not magically become qualified to engage in electronic discovery activities simply be calling itself an electronic discovery provider, nor is an organization automatically disqualified simply because it is a law firm or a client. Rather, it is a question of expertise, experience, resources, and the like.

What the question really shows is the disdain attorneys tend to have for people with technical training, no matter how experienced or well educated they are. I mean come on now, would that same attorney ask why his IT or secretary couldn’t draft a motion for summary judgement or a restraining order?

The fact is that most attorneys think if you don’t learn it in law school it isn’t professional and anyone should be able to do it. As the dean of one prestigious law school once told Browning Marean and I when we approached him about doing an e-discovery course: “we train architects not carpenters.”

Which is why in New York or Chicago or LA first-year associates fresh out of law school make 2-3 times as much as IT and lit support professionals with 10-15 years’ experience. And why Michael Arkfeld continues to say that 95% of the attorneys in the country don’t understand ediscovery at all.

So is building a house too expensive when you get your inexperienced, unlicensed friends to do it for you? You bet. But get a professional carpenter, know what you’re doing when you negotiate with them, and you’ll get a great house.

The same principle applies here.

ACEDS 2016 Conference Is In New York City Next Week, and Certification Is Still Center Stage

April 14, 2016

From this weeks Advanced Discovery blog:

ACEDS is holding their 2016 E-Discovery Conference on April 18-20 at the Grand Hyatt hotel in New York. Hundreds of ACEDS members will be attending to advance their education and career.

ACEDS gathers top judges, industry leaders, attorneys, litigation support staff and vendors to deliver knowledge on trending e-discovery topics and offer solutions to the most pressing issues in e-discovery by exchanging that knowledge among speakers and attendees.



The conference does assume a basic level of e-discovery experience and knowledge and all the sessions are designed in open format with panelists who may have different opinions on breaking issues to express them freely. Peer networking is a major component as well, with free flowing discussions at a variety of social events.

Known as an interactive conference, attendees are encouraged to dialogue with all the speakers, and from groundbreaking jurists to hardworking paralegals, speakers and attendees at this event traditionally place a high value on community, collaboration and conviviality.

The Conference will feature:

In addition to efficiencies in current practice and grappling with cutting edge e-discovery issues, special sessions will be focused on employment, career advancement and emerging adjacent opportunities for technical legal people. Those special sessions begin on Monday with a CEDS Exam Prep Seminar taught by e-discovery expert Attorney Helen Bergman Moure followed by a seminar on Cyber Security for Legal Professionals and a Law Student Blogger session.


In addition, EDRM will be holding their Spring Workshop on Monday and more information on that event can be found on the EDRM web site at

The ACEDS program kicks off on Tuesday morning, and a full schedule can be found at  I’ll be speaking with Craig Ball in a program called “The EDNA Challenge: Part 2.”  We’ll update the analysis done in Craig Ball’s influential paper “E-Discovery for Everybody: The EDNA Challenge” by discussing how to handle e-discovery on a $5000 budget.  The program will demonstrate proven and cost-effective methods, as well as some creative new ways to do discovery on a shoestring budget.

Then, later that afternoon, I will be doing an “E-Discovery & Pro-Bono Workshop” with attorney Suzanne Clark, CEDS, Vice-President & Membership Chair of the Jacksonville FL ACEDS Chapter. We’ll discuss traditional pro bono activities as well as newer activity in education, the vendor world and small firms, finishing up with a call to action for new mentoring efforts in this much needed area.

So, if you’re New York next week, come by the Grand Hyatt. And, if you can’t make it, email me at so I can fill you in on everything we discussed!

Florida Bar Taking the Next Step In Certification for Technical Competence

April 12, 2016

 Recently on the Advanced Discovery blog, I mentioned my participation on the faculty at the 4th Annual U of Florida Law –EDRM EDiscovery Conference held on the campus of the U of F Levin School of Law campus in Gainesville.  Among the many great speakers pictured below you’ll recognize Craig Ball, George Socha of the EDRM, David Horrigan and Rene Laurens of KCura, Niloy Ray of Little Mendelson, Gavin Manes of Avansic, Julie Brown of Vorys, Sater, Seymour and Pease and Rene Martin Audet of Nuix among others.


Canaan Himmlebaum of Advanced Discovery was featured on a panel entitled Who Says TAR is Only for Litigation Giants.


I was on a great panel called Similarity Takes Us Below the Surface with Rene Laurens, Julie Brown and Michael Dalewitz.


But the highlight of the conference was the focus on technical competence with an exciting keynote address on that subject by Craig Ball which was introduced with an even more exciting piece of news by Michele L. Lieberman, the Alachua County Attorney who is also a member of the Executive Council of the City, County and Local Government Law Section of the Florida Bar and is Secretary of the Eighth Judicial Circuit Bar Association.


Ms. Lieberman announced that the Florida State Bar has modified and sent to the Florida Supreme Court for final approval a change to their MCLE standards which requires 3 hours of technical competence training for every three year CLE requirement cycle. Once approved, this will make Florida the first state to require that its attorneys receive continuous ongoing training in order to guarantee their technical competence.

Coming on the heels of the advisory opinion on technical competence by the California Bar, this requirement shows an ongoing awareness by the Bar community of the need for specific technical training.  Hopefully it will be the first of many Bars to recognize such a need and take the specific steps necessary to see that such competence exists in the legal community.

Kudos to the Florida Bar and their forward thinking position on this urgent matter.

Reports of the Death of Keyword Searching Are Greatly Exaggerated

April 1, 2016

This week on the Advanced Discovery blog I noted that several websites are recirculating a article, first released in January just before LegalTech New York, entitled And the Judges Say: It’s Time to Adopt New Legal Technologies.  It’s an excellent piece that was an introduction to a great LegalTech session with Judges Andrew J. Peck (SDNY), James C. Francis (also S.D.N.Y.), Elizabeth D. Laporte (N.D. Cal.), and Pamela Meade Sargent (W.D. Va.), in which the judges discussed what they are currently seeing in their courts regarding big data, analytics, e-discovery and other technologies.

However, part of the article is a quote from an interview with Judge Peck in which he says, “I think there’s just too much data to try and do it the old fashioned way. That’s whether you’re talking really old fashioned with eyes-on-everything for review, or the still-old fashioned in my view use of keywords.”

With all due deference to Judge Peck, who I respect and admire both professionally and personally, I’ve disagreed with that position in the past and I still disagree with it.  Here is my reasoning.

Much of the lack of confidence in keyword searches is laid at the feet (or pen) of Judge John Facciola, in the case of United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008), with his famous quote about going where angels dare to tread. But that’s not exactly what Judge Facciola opined in that case. He actually dismissed a defendant’s objection to the adequacy of keywords used by the prosecution and ruled that a party challenging the efficacy of an opposing party’s search terms must do so through expert testimony.

His hesitancy was about the Court itself undertaking the complexity of search in the identification and production of electronically stored information. To that specific point, he stated that “[w]hether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics. … Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread.”

To be fair, in an earlier decision, Disability Rights Council of Greater Washington v. Washington Metro Transit Authority, 242 F.R.D. 139 (D.D.C. 2007), Judge Facciola did, in fact, state that concept searching is more likely to produce comprehensive results and is more efficient than keyword searches. However there, as in O’Keefe, he questioned the litigants’ ability to demonstrate to him that the results were defensible.

I’ve noted numerous times in the past six months that despite the strong scientific evidence in support of TAR, numerous polls from organizations such as eDJournal and Kroll, as well as client surveys from firms such as Gibson Dunn and Norton Rose, show that the majority of people are NOT using TAR. And if they are not, it seems clear they must be using keyword searches.

This position was supported publicly last year at the Today General Counsel conference in New York by Gene Eames, Pfizer Inc.’s director of Search and Analytics in their Legal Division. Gene made it clear that he is strongly in favor of keyword searches IF the keywords and the results can be tested and validated. His point was that you use keywords to propagate a seed set for the eventual computer search, so why not use it on all the documents as a first pass.

Several people noted that pointing any search tool at all the data may be costly and inefficient. Co-chair David Kessler of Norton Rose Fulbright stated, “If I’m playing hide-and-seek with my kids and it’s my turn to seek, I’m not looking in the breadbox. They won’t be there because they don’t fit there.”  That comment prompted Gene to recall a discussion he had with a federal judge about the best way to proceed in a search, where he said if he returned from a meeting at the courthouse to find he had lost his keys he wouldn’t begin a search in Penn Station; he’d start in the lobby of the courthouse.

Everyone in the conversation did agree that the best practice is to bring some common sense to your search process. Technology is great, but it’s not an “Easy Button”, and the best technology for your project depends on a number of variables, including budget, time constraints and search needs. As Maura put it, “TAR is a process, not a product.”

The point about keywords was brought home again several weeks ago at the ASU Arkfeld EDiscovery Conference in Tempe, when I spoke on a panel about keyword searches to an SRO crowd. As I said in my recap of that session, “…keyword search is far from dead and is probably still the most common search method used by eDiscovery staff. TAR may be up and coming, but keyword search is still king.”