Essential Discovery
Essential Discovery - managed-review

Quality Control Protocols

A defensible quality control process is key to a successful review. Essential Discovery has expertise in both designing and implementing quality control protocols. Our quality control protocols will give you the confidence that privileged documents are accurately tagged, consistency of coding amongst the review team is ensured, and reviewers that are having difficulty with speed and/or quality are identified. The end result is a superior work product at a lower cost.

Essential Discovery can implement any of the following quality control protocols at the request of the client. We strongly recommend utilizing at least one of the below quality control protocols for your review.

Method #1

Random Sampling/Reviewer Based Quality Control

  1. Create a one-out-of-ten random sample of reviewer documents/batches.
  2. Create mandatory protocols to review all responsive, potentially privileged documents that aren’t tagged privileged in a reviewer’s batch.
    1. Other mandatory reviews of specific tagging sequences (further review, tech issue, etc.) can also be included.
  3. Once a reviewer consistently receives a passing grade, then sample 1 out of 25 documents produced by that reviewer.
  4. This process can be coupled with a grading system to graduate reviewers to levels that require less sampling.

Method #2

Keyword Search, Utilization of Technology, and Re-Review of Presumptively Privileged Documents that Were Marked Responsive but Not Marked Privileged

  1. Create queries based on identified hot documents and key terms.
  2. Run searches for presumptively privilege documents tagged both not privilege and responsive.
    1. Depending on the volume of hits, we can modify the “potpriv” searches to be either more or less inclusive.
  3. Utilize advance tools of platform.
    1. Run searches for similar docs/dupes/near dupes to confirm if one of the documents has been marked privilege or confidential so that all of the documents have been marked privilege/confidential.
    2. A similar approach for redactions is used.
  4. The utility of this process is two-fold:
    1. Identify documents reviewers may have missed during the initial first level review that are key to the case.
    2. Add an extra layer of privilege protection.
  5. The tradeoff in utilizing this approach is that it may decrease visibility into reviewer performance; however, use of a history tool should highlight if a reviewer is repeating key mistakes.

Method #3

Mixture of Random Sampling and Keyword Search

Another and more thorough approach to the quality control process would be to utilize random sampling and keyword search together.

  1. This gives an overall feel of the general quality of each reviewer, while focusing in on key issues and privilege.
  2. Tradeoffs with this approach are the possible volume of quality control, and the added costs of conducting both levels of scrutiny.

Method #4

Inconsistent Tagging

In addition to the three approaches mentioned above, Essential Discovery can run an inconsistent tagging search over the population of reviewed documents.