Time Nick Message 12:25 dotplus seems like it could be useful. 12:26 dotplus There is one likely (serious, negative) side effect that strikes me. But that's one that seems to be becoming very prevalent all over our data-driven world and that is the misinterpretation of statistics. 12:30 dotplus I need to organise my thoughts on this properly and write it up, but the basic idea (modulo early morning brainfail) is: that the more we have and rely on data to inform our understanding and decisions about what is true, which seems to be a good thing and helpful, the greater the Significance (for our Understanding) of inaccurate/incomplete data. And given that for almost all interesting questions data is *necessarily* massively incomplete, ... 12:30 dotplus ... this is a big problem for our understanding. 12:31 dotplus s/statistics/data/ 12:36 pdurbin dotplus: yes, but that's a danger I'm willing to risk :) 12:38 dotplus Once upon a time I was working for 1 team of ~3 team IT group (unix/windows/hardware + a couple of partial FTEs which didn't fit well into any group). Likely any good IT org, we used a ticketing system to help ensure requests didn't get dropped. $boss was aware that (some of windows and hardware were ... "underperforming"). $grandboss (above the IT org) wanted to use the ticketing system to measure "productivity" and prove/disprove the claim ... 12:38 dotplus ... to decline/support dismissal. 12:41 dotplus *we* enlightened folks can see half-a-dozen problems in an eyeblink with attempting to interpret ticketing system data to determine whether someone is a useful IT employee or not, but it's not that uncommon to do so. 12:41 bene_ that's not a problem of bad/incomplete data though 12:42 bene_ that's misusing the data you have 12:42 dotplus not entirely, but bad or incomplete data exacerbates the problem and increases the likelihood 12:42 dotplus of misuse 12:43 bene_ i don't see the trend of increasing our reliance on data-driven decision making being bad in that case though 12:43 bene_ in a previous decade it would have been "fire ralph. my gut says he's slower." 12:43 bene_ is that any better? :-) 12:44 bene_ i think you're generally right though 12:44 bene_ the data-driven approach is not a panacea 12:44 bene_ and it comes with its own traps and pitfalls 12:45 bene_ the metrics you choose to measure, especially as they relate to human performance, will be gamed, intentionally or not, by those same humans 12:45 dotplus not from grandboss. it would have been "fire those who work for you who are slacker(s)" i.e. delegation to judgement. which in this case would have been good. 12:47 dotplus yes, the metrics you choose to measure will be gamed and the interpretations you garner from those metrics will influence the future. 12:50 dotplus pdurbin: I'm not arguing against "My ranking" or whatever. I think having a simple, convenient way to say things like "it looks like way more people care much more about adding support for markdown than restructuredtext" would be awesome. I'm just warning (Cassandra style?) that this will be abused intentionally or ignorantly. 12:51 dotplus not least because, for anything but the simplest questions, interpreting the data will be *difficult* 12:52 dotplus difficult as in a "Hard Problem" 12:56 dotplus bene_: And in some ways, I would say that "the metrics says Ralph is a slacker, fire him" is at least as bad as your "my gut says.." situation, because there is a (disingenuous?) call to (bogus) authority. "The data shows he's a slacker" when there are usually going to be other possible explanations that are less likely to be examined because the data/metrics/presentation/experiment don't highlight certain (relevant) factors. 12:57 dotplus at least in the "my gut says" case, there is no pretence of objectivity 13:32 bene_ the end result is the same for ralph though :-/ 13:32 bene_ i wrote that line about 30 minutes ago but didn't hit enter because it's tangential to your point i think 13:33 bene_ the call to authority thing is an interesting point 13:34 bene_ i wonder if our (the US) general litigiousness has played a part in the data-driven part of HR decisions 13:35 bene_ i think we're more in agreement than not 13:35 bene_ to speak to your original thesis though, i think in this instance it's more a case of not measuring the right thing rather than having inaccurate/incomplete data 13:36 bene_ the data says ralph is slower to close tickets/closes fewer tickets/etc 13:38 bene_ whether or not that's the right metric seems more like an alternate failure mode for the data driven approach to me 13:39 bene_ i was making and drinking coffee in the 30 minute window before hitting enter :-) 14:03 dotplus yes, I'm not addressing whether it sucks for Ralph. It might or it might be just what he needs:) 14:05 dotplus I agree that the general litigiousness and CYA mentality of "developed" (organizational?) culture, not just USian, is a contributing factor. 14:08 dotplus I also agree that "not measuring the right thing" is also a major problem. And I think it partly results because it's usually difficult (impossible?) and almost always (too?) expensive to determine what the "right thing" is. 14:09 dotplus And yes, I think we're more in agreement than not 14:10 dotplus except about $beverage 14:10 * dotplus pats his teapot 17:01 hydrajump this is very cool http://casual-effects.com/markdeep/ 17:02 hydrajump "Markdeep is a technology for writing plain text documents that will look good in any web browser. It supports diagrams, common styling conventions, and equations as extensions of Markdown syntax."