Time to "Kill the Rankings"

Just a few days after their 6th Annual Electronic Discovery Survey, George Socha and Tom Gelbmann announce on eddupdate.com that “the time has come … to kill our rankings.”

But all is not lost, Socha and Gelbmann also announce that they “intend to replace those rankings with an improved way to assess providers and their capabilities.”

The post provides some history about their survey and why it was attempted in the first place. Six years ago, the e-discovery market was a very spooky place, and it was difficult to find any information on vendors and providers. That sounds ridiculous today with the overwhelming daily deluge of e-discovery information we’re all inundated with.

Six years ago, however, George and Tom stepped up to the task of providing a practical guide to the world of e-discovery with their survey and it has been one of the most influential assessments of the industry.

I had the opportunity to participate in this year’s survey, and I can say from first-hand experience that the survey is extremely thorough and well-organized.

Now George and Tom, instead of resting on their laurels, have decided to change their direction because they are aware that their results a) “can affect the share prices of publicly held companies” and b) “they are sometimes the most important factor in determining which provider is selected to take on a project or deliver a software program.”

I have the utmost respect for both gentlemen in this decision. They have ALWAYS presented their survey “as objectively and fairly as possible,” but alas they realize that their lists are sometimes used as the ONLY factor in deciding what vendor or service provider to use, even with their stern warnings cautioning against this.

I  sincerely admire this statement:

We are hearing, as well, that consumers are using them as a substitute for the work they should be doing themselves — analyzing consumer needs and assessing what and whose services and software might meet those needs.

Everyone likes lists, and in my experience, lawyers love them. Regardless of how much experience I have with an e-discovery vendor, or how much research I put into using a particular vendor, if they are not on the Socha-Gelbmann list, they are suspect in the eyes of every lawyer I have worked with.

To be fair, this reflects the respect that the Socha-Gelbmann survey has garnered, and it’s always easiest to assess an industry with a list (e.g. Fortune 500 or AmLaw 100).

But just because an e-discovery service provider is not at the top of the list doesn’t mean that they are not the best selection for the project at hand in terms of flexibility, cost, capacity and experience. Unfortunately, proposing an appropriate vendor for a project usually requires a good amount of experience in the industry which many legal professionals do not have.

I am excited to see what George Socha and Tom Gelbmann come up with next. They promise “to continue to publish a high level view of key industry trends, market size and providers and their capabilities in Law Technology News.”

Link to story.

UPDATE: Sharon Nelson posts her thoughts on the demise of the Socha-Gelbmann Survey as we know it.