by now i would imagine that most people keeping an eye on the industry have noticed a bit of bad blood emerging between panda and av-comparatives...
luis corrons' post on the panda blog seemed a little like holding someone's feet to the fire - something i've done on occasion so on that level i can't really fault them (and really, sometimes people's feet do need to be held to the fire)... however, there are some very important tricks to doing this successfully... one of the most important being to not have a financial motivation to make the other party look bad - i get away with the things i say in part because i'm not actually in the anti-malware industry, when i say someone's doing something bad there's little or no possibility for my actions to be attributable sour grapes... luis, who represents a product that has apparently been tested by av-comparatives in the past has a financial motivation to call av-comparatives methods into question if that product didn't fare as well as one might hope in the past... perhaps his characterization of the testing at av-comparatives is correct, perhaps it isn't, but a panda representative is not in the best position to be pointing it out...
robert sandilands over at the authentium blog recognized this and threw in his own two cents coming from the perspective of one whose product has never been tested by av-comparatives... unfortunately, robert opted for the personal approach, subtracting the problem element that luis encountered but adding nothing good to replace it... personally, i'm not the least bit interested in his reaction to meeting andreas clementi, there's nothing of substance or value to be found in insinuations...
you see, the second important trick to pulling off holding someone's feet to the fire is to cite specific examples of problems, and back them up with evidence if at all possible so that your criticisms are more than just baseless claims (or worse, simple character assassination/defamation)... luis tried at least to point to a specific problem when he brought up the issue of paid services provided by av-comparatives, but botched the job by giving the apparently misleading impression that av-comparatives charges av vendors money to be included in their tests (which would certainly call the independence of av-comparatives into question if it were true)...
of course, even if the fees charged by av-comparatives are for things other than inclusion in the test they're still in an awkward position... it's very difficult to profess independence from av vendors when there's any kind of relationship with them... look at virus bulletin, sister company to sophos, which suffered questions about it's independence for years... those concerns fade away a little bit every time sophos fails to achieve a vb100 award but what a dysfunctional non-competitor relationship that winds up being; one company can only do well at the expense of the other... andreas clementi did well to clarify what av-comparatives' fees were for in his post about fee structure, more transparency may not dispel doubts but it at least gives people the information they need to judge for themselves, but i fear that so long as av-comparatives derives any financial benefit directly from members of the industry it's supposed to provide metrics for, the independence of av-comparatives and by extension the impartiality and validity of those metrics will be at least somewhat in question...
andreas' initial reaction to the panda blog post was pretty obviously written in the heat of the moment (which is perhaps why the post has been removed) but there are some issues raised that deserve attention, not the least of which being the apparently materially false statements made by panda, as well as how significant any specific problems with the testing at av-comparatives (should they actual exist) are...
(as a side note, this has highlighted a problem with using google cache to link to historical items as both cached documents appear to differ from what was originally seen)
0 comments:
Post a Comment