Time  Nick         Message
13:05 pdurbin      hmm, i wonder how easily i can install python 2.7.3 on centos 6
13:11 pdurbin      On linux SUSE or RedHat, how do I load Python 2.7 - Stack Overflow - http://stackoverflow.com/questions/4149361/on-linux-suse-or-redhat-how-do-i-load-python-2-7
13:11 pdurbin      "make install will damage the system Python. Use make altinstall instead." :)
13:31 Itkovian     or use EasyBuild :-D
13:43 pdurbin      Itkovian: yeah? i was thinking of easybuild more for the whole cluster. for use with "modules"
13:43 Itkovian     surem but you can just install a single software package such as python as well
13:43 Itkovian     just clone and try on your local system
13:44 Itkovian     it's not going to install stuff where it overwrites other stuff
13:49 * pdurbin    joins #easybuild
13:50 pdurbin      looks like there's a quick start: http://hpcugent.github.com/easybuild/
13:51 pdurbin      Hearing on "The State of Research Infrastructure at U.S. Universities" -- http://casc.org/papers/THDJr_CongressionalTestimony_Feb10.pdf
13:57 pdurbin      crimsonfubot: lucky easybuild yum repo
13:57 crimsonfubot pdurbin: https://github.com/hpcugent/easybuild/wiki/FAQ
14:33 pdurbin      whorka: try `yum install environment-modules`. it's in base on el6
14:37 pdurbin      hmm, lots more people at https://github.com/crimsonfu?tab=members than http://crimsonfu.github.com/members
15:19 pdurbin      crimsonfubot: lucky mpfs wikipedia
15:19 crimsonfubot pdurbin: http://en.wikipedia.org/wiki/Multi_Path_File_System
15:39 GitHub38     [crimsonfu.github.com] pdurbin pushed 1 new commit to master: https://github.com/crimsonfu/crimsonfu.github.com/commit/716e2471740fd37f19cba26d6db9fad4441edf13
15:39 GitHub38     [crimsonfu.github.com/master] fetch the latest logs - Philip Durbin
15:40 pdurbin      new! http://irclog.perlgeek.de/gluster/today
15:57 pdurbin      ventz: http://irclog.perlgeek.de/gluster/2012-10-05#i_6038551
17:03 ventz        Anyone who is at Harvard/around physically, if you like geeking out, come by Wednesdays for geeknight (which has turned into lunch for now)
17:03 ventz        (join #geeknight for more info)
17:29 pdurbin      ventz: nice public service announcement :)
17:34 ventz        :)
17:34 ventz        hehe
17:34 ventz        if you weren't in #geeknight, i probably would not have sent this here
17:34 ventz        but i figured you approve on some level :)
17:49 pdurbin      i do approve. and may geeknights spread to every land
17:49 agoddard     ohai
17:50 pdurbin      agoddard: howdy, stranger
17:50 agoddard     pdurbin: now I'm "that guy" who just shows up when he wants to know something ;)
17:51 agoddard     pdurbin: or, that super busy guy who's always getting trolled by his todo list ;)
17:51 agoddard     I got a RAID / LVM question I thought I'd throw out there..
17:52 agoddard     I'm setting up a 24 disk array (500GB NL-SAS 7.2k disks) which will end up carved up as logical volumes through LVM on the host
17:52 pdurbin      magoo: *cough*
17:53 agoddard     I'm trying to decide if I should RAID-10 the array and present one big disk (RAID10 is kinda my instinct generally) or whether I should carve out a few RAID 10's, OR, whether I should even carve out a few RAID5's.. (I probably won't do one big RAID5)
17:54 magoo        well hey guys!
17:54 agoddard     I've been reading a bunch about high sequential reads being kinda useless in this use case, but then I read something else that says the more disks the better, just rock a giant raid10 (basically everything else here is RAID10 'cause of that)
17:54 agoddard     magoo: hey hey :)
17:54 magoo        don't need anything... just making sure everyone is having a great day ;)
17:56 pdurbin      agoddard: what's the use case?
17:57 pdurbin      magoo: you help us keep it friendly :)
17:57 agoddard     the LVM volumes will end up as data volumes for VMs - a lot of solr, mysql, elasticsearch and maybe some hbase stuff
17:58 pdurbin      VMs? JoeJulian likes VMs. so does comptona
17:59 pdurbin      i assume raid 10 would be the fastest but what do i know
17:59 comptona     that I do
17:59 comptona     beats racking new servers for every little thing
18:01 agoddard     and the i/o benefit of 24 drives would be worth it (as opposed to 2x RAID10's of 12 drives, both of which are added to the Volume Group)?
18:01 agoddard     oh, VMs :) ya. +100000
18:04 pdurbin      i'm not much of a storage wonk. you might want to ping our internal mailing list and report back the answer
18:04 pdurbin      whorka knows storage but he's idle, looks like
18:05 pdurbin      agoddard: could pop in #gluster. those guys know storage
18:05 agoddard     pdurbin: sweet, will ping rcops, I'd ask in gluster but I'd be worried they'll be like "lul raid wut!" ;)
18:08 pdurbin      JoeJulian: they'd never say that
18:10 agoddard     rackerhacker: ICQ lulz.
18:10 rackerhacker yeehee
18:10 rackerhacker worst of all, it's a guy who is probably trying to hire me :/
18:10 pdurbin      agoddard: openstack cluster? :)
18:12 agoddard     rackerhacker: ha!
18:13 agoddard     pdurbin: yep, I had enough of my cloudstack experiment so thought I'd actually sit down and read all the OS user guides, pumped about it now. Built a test cluster, demoed it to the CEO and a scientist who wants to start using it straight away 0_O
18:13 pdurbin      westmaas: ^^
18:14 pdurbin      agoddard: cool, i'm sending all my tickets to you. so glad you have RT access now
18:14 agoddard     the test cluster is just an all-in-one machine, but the prod cluster I just started on is 1 controller node, then 1 cinder node (with this array) and then the project to move 216TB of gluster to swift
18:14 agoddard     pdurbin: haha.. sweet, we should join clusters :)
18:16 westmaas     agoddard: pdurbin: nice!
18:16 westmaas     rackerhacker: trying to hire you?!
18:16 pdurbin      agoddard: well, even i've set up an openstack all in one machine
18:17 agoddard     pdurbin: I wouldn't be surprised if your laptop was an openstack all-in-one machine
18:17 agoddard     ;)
18:17 pdurbin      westmaas: i know. who would want to hire rackerhacker
18:17 westmaas     haha
18:18 agoddard     protip, don't present a new cluster idea to the CEO based on a little demo instance, when they get excited it seems to introduce unnecessary pressure :-/
18:18 rackerhacker westmaas: it's a former racker who i'd never work for anyway :)
18:18 westmaas     agoddard: its fine don't worry about it?
18:18 westmaas     rackerhacker: haha
18:18 westmaas     agoddard: did I mention we sell installation and support services for openstack clusters? :P
18:19 pdurbin      agoddard: moving *off* of gluster for VM storage?
18:19 agoddard     westmaas: I think I rememebr..u at Rackspace?
18:20 westmaas     agoddard: yep yep
18:20 westmaas     but thats not my group
18:20 westmaas     so I'm happy to provide the services of rackerhacker for free
18:20 agoddard     pdurbin: we don't do any VM storage on gluster at the moment, just a whole ton of storage for a giant image repository
18:20 rackerhacker westmaas: that doesn't sound right
18:20 westmaas     rackerhacker: don't worry about it
18:20 agoddard     haha
18:23 pdurbin      agoddard: ok
18:23 agoddard     westmaas: /me checks budget.. curses, makes note to talk to boss about budget
18:23 westmaas     lol
18:23 westmaas     anyone making it to OS summit this year?
18:23 westmaas     its in a week!
18:23 westmaas     or so
18:23 agoddard     pdurbin's boss is I think
18:23 westmaas     managers, huh guys? buncha jerks
18:24 pdurbin      agoddard: that sounds right
18:24 westmaas     hmm yes its in san diego, I better go
18:24 westmaas     pdurbin: want me to say hi to him or stay away?
18:25 pdurbin      westmaas: please!
18:25 pdurbin      i wanted to have something for him to demo. something openstack-y :(
18:27 * agoddard   will be back in 5, if he's not back in 10, he's been crushed by a heavy server
18:35 * agoddard   made it out alive
18:35 pdurbin      phew
18:36 pdurbin      migration-status.pl > status/raw.mdwn
18:36 pdurbin      i love ikiwiki
18:38 agoddard     :D
18:39 larsks       Aaaaa, I went to install ikiwiki to look at it and it's pulling in like 90% of CPAN.  My system is being perled!
18:39 larsks       Could be worse.  Could be RT.
18:40 agoddard     I'm one of the people who hates JIRA so much he uses it.. :-/
18:40 larsks       Oh, I like RT; it just has a dependency list a mile long.
18:40 agoddard     I gave up fighting against it, and now I (don't really) like it :)
18:47 pdurbin      larsks: try my rpm: http://wiki.greptilian.com/ikiwiki
18:47 larsks       pdurbin: I *am* install via an RPM (ikiwiki is included in F17, which is what I run on my desktop).
18:47 pdurbin      ok
18:47 larsks       It just wants a bunch of perl modules :).
18:48 pdurbin      it's full of perly goodness
18:49 pdurbin      larsks: my setup, if you're interested: http://git.greptilian.com/?p=wiki.git;a=blob;f=wiki.setup
18:49 larsks       Thanks, I'll take a look.
18:58 JoeJulian    wrt me liking VMs, Keeping services on separate virtual hosts allows me to do system and hardware level maintenance without service interruption.
18:59 pdurbin      JoeJulian: sing it
19:00 pdurbin      spotted; "ATLAS is a nasty bitch, it does auto-tuning"
19:01 comptona     JoeJulian: plus, if people insist they need root to run their services, then you can give them a VM to screw up without bothering anybody else
19:02 pdurbin      comptona: but then they break their vm and bring you the pieces
19:03 comptona     at least it's fewer pieces than if they break a machine other people use too :-/
19:03 pdurbin      oh, people break the whole cluster too
19:03 comptona     !
19:04 larsks       ...this is why we're playing with OpenStack.  We those people to be able to deploy their own vm and services without requiring help from us.
19:04 pdurbin      it's ok. it's in the name of science
19:04 comptona     yeah, I just set up openstack at $work and it's getting super popular
19:05 JoeJulian    If the vm is broken, I just delete it and let puppet rebuild it from scratch.
19:10 pdurbin      JoeJulian: yeah, i about to suggest to this user that we rekickstart this sick vm. we use cobbler and puppet for that
19:10 pdurbin      don't overwrite the system python, kids
19:14 pdurbin      s/i/i'm/
19:24 pdurbin      agoddard: http://irclog.perlgeek.de/gluster/2012-10-05#i_6039201 ask away, i'd say :)
19:30 boegel       pdurbin: don't overwrite the system Python kids, use EasyBuild instead! ;-)
19:30 pdurbin      boegel: trying! is this the "quick" demo?
19:31 boegel       pdurbin: :D
19:32 boegel       pdurbin: it's quick in the sense that you only need three commands
19:39 pdurbin      ah
19:43 boegel       pdurbin: ;-)
19:44 boegel       pdurbin: it's something like execute the commands, have a coffee (or two), and EasyBuild did the work for you
19:50 pdurbin      i'm on my 5th coffee
19:50 pdurbin      Process 24999 detached
19:51 pdurbin      WARNING: Build exited with exit code 1. 2277 possible error(s) were detected in the build logs, please verify the build.
19:51 pdurbin      :(
19:52 pdurbin      18 MB log
19:54 pdurbin      boegel: zips down to 556 KB. have an email I can send this to?
19:55 boegel       pdurbin: is this the full log?
19:56 boegel       pdurbin: those detected errors probably all are false positives
19:56 pdurbin      18 MB sounds pretty... comprehensive for a log
19:56 boegel       pdurbin: and yes, I have an email :)
19:56 boegel       pdurbin: it's a debug log :)
19:56 boegel       pdurbin: kenneth . hoste @ gmail
19:58 pdurbin      Subject: easybuild-ATLAS-3.8.4-20121005.123157.log.gz
19:58 pdurbin      boegel: sent
20:03 boegel       pdurbin: oh, build failed :-/
20:04 boegel       that's not good
20:04 boegel       let me check the log
20:04 pdurbin      i didn't think so
20:05 pdurbin      thanks
20:05 boegel       pdurbin: what processor are you building on?
20:05 boegel       pdurbin: Intel Sandy Bridge, by any chance?
20:05 pdurbin      qemu something..
20:06 pdurbin      QEMU Virtual CPU version (cpu64-rhel6)
20:07 boegel       pdurbin: hmm
20:07 boegel       pdurbin: the real issue is this: fatal error: atlas_pthreads.h: No such file or directory
20:08 boegel       pdurbin: but I'm not sure how that even can happen
20:10 boegel       pdurbin: I think ATLAS is having a hard time handling your Virtual CPU...
20:10 pdurbin      boegel: i swear i didn't fake the logs ;)
20:11 boegel       pdurbin: I'm seeing stuff like " dmm.c:1750: Error: bad register name `%r8' " as well
20:12 boegel       pdurbin: I think building ATLAS on a system with a virtual CPU is the real issue here...
20:13 pdurbin      boegel: want to make me an "easy python 2.7" instead? :)
20:13 boegel       pdurbin: well, you're going to need a toolkit first
20:13 pdurbin      oh
20:14 boegel       pdurbin: do you have access to Intel compilers?
20:15 boegel       building Python will requires BLAS and possibly LAPACK support
20:15 boegel       that's not going to work without a toolkit
20:15 pdurbin      i thought building python required make and gcc
20:17 boegel       pdurbin: ok, wait, if you exclude numpy/scipy, you can build Python with EasyBuild without a toolkit
20:17 boegel       pdurbin: which is not it's intended use, but it will work
20:18 pdurbin      boegel: might make a faster demo
20:18 boegel       pdurbin: well, we really want to encourage people to use toolkits
20:19 boegel       pdurbin: because that way the software you build is as independent from OS-provided stuff as possible
20:19 pdurbin      hmm. ok. not sure what an easybuild toolkit is
20:19 * pdurbin    has not read the docs
20:20 boegel       pdurbin: it's basically a compiler with an MPI library and math libraries (BLAS, LAPACK, ...)
20:20 boegel       pdurbin: it's everything you need to build most scientific software
20:21 boegel       pdurbin: the evil thing is that you need to built the toolkit first, i.e. build GCC, ATLAS (which provides BLAS, LAPACK, ...)
20:21 boegel       pdurbin: there are replacements for ATLAS though, which may maybe fit better in a quick demo
20:21 boegel       pdurbin: e.g. we could use GotoBLAS+LAPACK to replace ATLAS
20:22 pdurbin      boegel: gotcha. making more sense now. ATLAS truly is a dependency
20:23 boegel       pdurbin: if you want to build scientific stuff, yes
20:23 boegel       pdurbin: I can explain you how to get Python built with a toolkit, if you like
20:23 pdurbin      and we would, in the general case
20:23 boegel       pdurbin: you just need to copy and adjust a simple text file (easyconfig file)
20:23 boegel       pdurbin: and then run eb with that file provided
20:24 pdurbin      boegel: that might be useful. right now i usually go check fedora or rpmforge for newer versions of software
20:25 pdurbin      the ikiwiki rpms i was pointed out to larsks earlier are just rebuilds of fedora srpms
20:25 boegel       pdurbin: ok, here's how
20:26 boegel       pdurbin: 1) copy the $EBHOME/easybuild/easyconfigs/p/Python/Python-2.7.3-goalf-1.1.0-no-OFED.eb file to $HOME/Python.eb
20:27 boegel       pdurbin: then, edit it to use a dummy toolkit ("toolkit={'name':'dummy','version':'dummy'}" ), and remove the pkglist specification (or atleast remove numpy/scipy in that list)
20:27 boegel       pdurbin: then, run eb while supplying that file
20:27 boegel       pdurbin: EasyBuild will download the Python sources, built it, install it and create a module for it
20:28 pdurbin      boegel: that's it?
20:28 boegel       pdurbin: that's it
20:28 boegel       pdurbin: there's a reason it's called EasyBuild
20:28 boegel       pdurbin: let me know if you run into problems there...
20:29 GitHub41     [crimsonfu.github.com] pdurbin pushed 1 new commit to master: https://github.com/crimsonfu/crimsonfu.github.com/commit/4f0cf91320264de7990ca90f8aac5ac196445510
20:29 GitHub41     [crimsonfu.github.com/master] save the easybuild fu - Philip Durbin
20:29 boegel       pdurbin: I haven't tested this right now, but it should work
20:29 pdurbin      boegel: will do. later. thanks!
20:30 * boegel     tries it on his end to verify
20:30 pdurbin      heh
20:32 boegel       pdurbin: you can even drop the "--robot X" part of the command, that's only required to resolve dependencies, and Python doesn't have any
20:32 pdurbin      ok
20:32 boegel       pdurbin: so, the command would look something like "eb Python.eb"
20:32 boegel       pdurbin: easy, huh? ;)
20:32 pdurbin      and -ld was what i used to see some output, to know eb is alive
20:33 boegel       pdurbin: well, there are other ways to check it's alive, but yeah, -d for debug info, -l for logging to terminal
20:33 pdurbin      yeah, i found strace useful ;)
20:34 boegel       pdurbin: top works as well :P
20:34 boegel       pdurbin: or "ps axfu"
20:34 pdurbin      :)
20:35 boegel       pdurbin: just tested, instructions work
20:35 boegel       pdurbin: Python builds fine with a dummy toolkit, i.e. using the system compiler
20:35 boegel       pdurbin: assuming you have a compiler installed :)
20:36 pdurbin      i do. even intel. somewhere
20:38 pdurbin      pedmon was in this channel the other day. he went to IDF a few weeks ago
20:38 boegel       pdurbin: if you have Intel source tarballs, you can make sure EasyBuild finds them, and then make it install the ictce toolkit
20:38 pdurbin      http://www.intel.com/idf
20:39 pdurbin      ok. i don't know what this is but i'm sure pedmon does
20:42 pdurbin      29 more VMs to migrate. have a good weekend, all