Time Nick Message 15:17 bencomp pdurbin sent me 15:18 semiosis large marge sent me 15:18 pdurbin lolz 15:18 pdurbin bencomp: well, you know way more about logstash than I do: http://irclog.iq.harvard.edu/dataverse/2014-08-01#i_11801 15:23 bencomp I think I've found the use of aligning logs 15:24 pdurbin not even sure what that means 15:25 pdurbin searchbot: lucky logstash align 15:25 searchbot pdurbin: http://serverfault.com/questions/604653/problems-configuring-logstash-for-email-output 15:26 bencomp "use case for aligning log files"? 15:27 pdurbin yeah 15:40 bencomp is Apache Brooklyn on topic too? (just found out that it exists) 15:41 pdurbin searchbot: lucky apache brooklyn 15:41 searchbot pdurbin: http://incubator.apache.org/projects/brooklyn.html 15:42 semiosis grok is a library built on top of regex for parsing strings 15:42 semiosis lets you build up very complex patterns out of simpler component patterns 15:42 pdurbin oh, this is a better site: http://brooklyn.incubator.apache.org/ 15:42 pdurbin "Simplified deployment and runtime management of enterprise-grade applications" 15:42 pdurbin bencomp: yes. on topic 15:43 semiosis here's the base patterns, you can see how they go from simple to complex: https://github.com/elasticsearch/logstash/blob/master/patterns/grok-patterns 15:50 bencomp pdurbin: Glassfish grok patterns: https://gist.github.com/bencomp/9b99cee2720d6233c8a1 15:52 semiosis nice! 15:56 semiosis i wrote these a long time ago, they probably dont even work anymore... glusterfs: https://gist.github.com/semiosis/1499710 & ModSecurity: https://gist.github.com/semiosis/1346387 16:04 bencomp semiosis: I bet you know logstash even better than pdurbin thinks I do - would you recommend running it on a production server next to or inside glassfish? 16:05 semiosis hah 16:05 semiosis i've been away from it so long 16:06 semiosis i used to run it on prod alongside other stuff, in its own jvm. but i didnt maintain it and now i need to rip it out & start over 16:06 semiosis not sure about running it in glassfish. that might be challenging 16:06 semiosis though possible in theory 16:07 semiosis if i were doing it over today, i'd look into using something like this: https://github.com/ryantenney/log4j-redis-appender 16:07 semiosis and writing out structured log messages to redis direct from my java apps 16:07 semiosis then having a logstash collector/processor consume from redis 16:08 semiosis or another message broker of your choice 16:08 semiosis logstash supports many 16:08 semiosis if you can modify your apps to write structured logs that's the best. grok should only be used as a last resort imho 16:09 semiosis afk a bit 16:10 bencomp ooh, that's some useful advice - Im really a beginner at log management (I currently rsync the various log files and try to get them into elasticsearch using logstash, and then visualise them with kibana - on my laptop. that doesn't work very well) 16:11 bencomp it wouldn't necessarily need to run inside glassfish 16:14 * pdurbin doesn't know logstash at all. the guy who used to sit next to PaxIndustria showed it to me once