Time  Nick        Message
15:21 pdurbin_    takes a while for request tracker to pull down all its dependencies during installation
15:26 agoddard    any of y'all using openstack or cloudstack?
15:31 pdurbin_    hoping to stand up openstack some day!
15:34 pdurbin     agoddard: we had talked a bit about openstack vs. cloudstack at http://irclog.perlgeek.de/crimsonfu/2012-04-04
15:35 pdurbin     "14:57 agoddard  my thoughts with cloud vs. open at the moment are basically - we need something with a nice API, preferably one that someone's already written a  knife plugin for, and we need something that'll rock iSCSI storage for our VMs"
15:36 pdurbin     agoddard: what's your latest thinking?
15:37 agoddard    ya, basically that still. I did some research into both last night, the key point is still the storage stuff
15:37 pdurbin     so sjoeboo was kinding with iSCSI a bit yesterday
15:37 agoddard    it looks like with OpenStack, I need to create an iSCSI target on a node even if I have hardware iSCSI, which kinds sucks
15:38 pdurbin     kinding? dinking. sheesh
15:38 agoddard    with CloudStack, I can provision LVMs directly, backed by iSCSI.. but only with XenServer as a hypervisor
15:39 pdurbin     right now KVM mounts NFS storage for VMs. but that storage is ultimately delivered over iSCSI. thinking about trying to hit the iSCSI directly...
15:39 agoddard    ya, that's what I'd like to do
15:39 agoddard    another option could be to use the NFS storage for the VM's system drive and then have the VM mount an iSCSI LUN directly when fast storage is needed
15:40 agoddard    that would cut out all the middlemen and persist across migrations, as long as all the networking was consistent across the hosts
15:41 pdurbin     hmm, in general we've been trying to have our VMs only use one disk image. simpler that way...
15:42 agoddard    ya, but if you have a machine you need quick disk on, there's not many other options.. maybe passing a local block device straight to it..
15:43 pdurbin     yeah. in general we haven't been using virtualization for high performance stuff. this may change, however, as we virtualize more and more...
15:43 agoddard    using XenServer as a hypervisor would be good for the iSCSI stuff, but then we have to deal with modified kernels and the fact that all the CPUs need to be the same, not only for live migration (which isn't that important) but to join the resource pool (which I think is kinda stupid)
15:45 pdurbin     is cloudstack working on better iSCSI support for KVM hypervisors?
15:46 agoddard    not sure, or whether openstack would be closer 'cause of the Dell support
15:46 pdurbin     westmaas: any thoughts on this? ^^
15:47 agoddard    like, the SANs we use are part of Dell's openstack reference architecture, even though you afaik can't actually use them as you would a traditional SAN in a traditional iSCSI/LVM setup
15:48 pdurbin     agoddard: do you have a link to the bit in the docs you're talking about?
15:48 agoddard    the direct access to block devices isn't always black and white anyway I guess, we had 6x better performance on MySQL when it was really busy (during data loading) on qcow2 backed KVM machines than we did on iSCSI/LVM backed XEN machines (I think maybe 'cause of a bug, but whatever it was it was slooow)
15:48 pdurbin     wow. 6x
15:49 westmaas    so you are looking at volumes attached to VMs?
15:50 agoddard    westmaas: ya, either directly provisioning machines using LVM backed by iSCSI (ideally and what we do with XenServer at the moment), or just thin provisioning machines and then attaching iSCSI LUNS directly to the guests I guess..
15:51 westmaas    agoddard: have you looked at any of the volume work being done? to be honest I haven't been involved in that enough yet to be intelligent, but I'm happy to find the people to get you any answers you need
15:52 pdurbin     i'm not so much interested in having volumes attached to VMs... more like... having KVM not access the VM images over NFS like we do now... but have KVM access the VM disk images over iSCSI instead
15:52 sjoeboo     here here
15:52 westmaas    there's some volumes work in nova right now, that let you boot straight from those volumens, or attach them after the fact
15:52 westmaas    gotcha
15:52 pdurbin     (not that i've even really looked for docs about this)
15:53 westmaas    OK, let me go find someone smarter about these things to get an answer for you
15:53 agoddard    if KVM is accessing the disk images over iSCSI, it'd need an API to the SAN, or it'd need to use LVM, then there's CLVM which doesn't support snapshots and gerrrgghhh
15:55 agoddard    westmaas: awsiq :) are you using something*stack at the moment?
15:56 westmaas    agoddard: I work at rackspace, we are rolling out openstack for our public cloud
15:56 agoddard    westmaas: aw nice!
15:57 westmaas    I might be biased :D
15:57 agoddard    westmaas: a lot of smart people have said good things about openstack :)
15:58 westmaas    and none about cloudstack!
15:58 westmaas    ok ok, maybe thats not true
15:59 westmaas    we are rolling out a custom volume service with the same API at rackspace, and I just need to find out the reasons why and see if that might be to get around some of the issues that you have
16:03 pdurbin     agoddard: please check your email for "Subject: kvm vm on nfs -> iscsi" from sjoeboo
16:04 pdurbin     i'm happily impressed that this works :) good job, sjoeboo (who just stepped out for a meeting)
16:05 pdurbin     basically, he did a live migration of a VM from one hypervisor with storage mounted over NFS to another hypervisor with the same storage mounted over iSCSI
16:06 pdurbin     so this will eliminate a single point of failure for us (the NFS server that's currently serving up the iSCSI storage)
16:06 agoddard    damn, siq!
16:06 * agoddard  keeps clicking "get mail"
16:06 pdurbin     now the question is... how do we eliminate the iSCSI storage array as a single point of failure
16:07 pdurbin     SEJeff_work: yes, yes, sheepdog, i guess. or something
16:07 pdurbin     maybe swift from openstack in the future, assuming i can get that working
16:07 agoddard    pdurbin: I'd be keen to work on a proof of concept with you if you want..
16:08 SEJeff_work pdurbin, Ironically, my friends at metacloud are not impressed with swift
16:08 SEJeff_work They are working on making Ceph talk directly to keystone so they can rip out Swift
16:08 agoddard    we were looking at moving from gluster to ceph a few months back, but never got back to it
16:09 SEJeff_work Not impressed with gluster?
16:15 agoddard    SEJeff_work: we had some early issues, probably resolved in the newer versions, we liked the look of ceph for what we're doing though
16:15 pdurbin     agoddard: i'll send you a picture of our whiteboard :)
16:15 agoddard    pdurbin: siq
16:16 SEJeff_work agoddard, I attended you a talk with Sage Weil (ceph author) last year and he said Ceph is good stuff, but that they don't backport the client code at all
16:16 agoddard    pdurbin: http://www.referencearchitecture.org/ < ha, it's from rackspace ;)
16:16 pdurbin     i guess to make the iSCSI storage more fault tolerant, i could just use linux software raid
16:16 SEJeff_work So if you want really good Ceph clients, you'll need very new kernels as he develops in the open in upstream kernel.org
16:17 agoddard    SEJeff_work: ah sweet, good to know. We had a call with some DreamHost folk about it, but they mostly did sales pitches :/
16:17 agoddard    pdurbin: in that ref. arch, under hardware is Dell MD3200i
16:17 SEJeff_work Yeah dreamhost uses Ceph + xfs from speaking with them
16:17 agoddard    pdurbin: we have a bunch of 3220i's w/ 1220 shelves
16:17 SEJeff_work agoddard, You know that Sage broke off mostly from dreamhost and started a company around Ceph, right? They are in downtown los angeles
16:17 SEJeff_work They still fund his work, but...
16:17 agoddard    SEJeff_work: ah nice, didn't know that
16:18 SEJeff_work http://www.internetnews.com/ent-news/ceph-inc-changes-name-to-intank-to-commercialize-open-source-storage.html
16:18 SEJeff_work agoddard, ^^
16:18 SEJeff_work Suse should buy that, to differentiate themselves from Redhat
16:24 agoddard    pdurbin: sjoeboo's email no arrive :(
16:36 pdurbin     agoddard: whoops. he only sent it to me. i'll forward with the picture of the whiteboard i just took
16:37 agoddard    pdurbin: thanks :)
16:55 pdurbin     agoddard: sent. to rcops-list. sorry. busy whiteboarding :)
17:10 agoddard    pdurbin: looks good!
17:52 pdurbin     we have 3000i's, not 3220i's but i assume they're similar
18:08 pdurbin     hee hee!  i just installed RT for the first time. i've always had to play with someone else's RT. i'm still talking about Request Tracker, of course
18:10 SEJeff_work Holy lots of perl module dependencies batman
18:10 SEJeff_work ditto for bugzilla
18:11 pdurbin     yeah, but in this vm i'm just doing it ironcamel's way. . . become root and worry not about where files are written
18:11 SEJeff_work cpan?
18:12 SEJeff_work That way works, but makes config management difficult
18:12 pdurbin     right. no rpms. but i think whorka might have RPMs for RT somewhere...
18:12 SEJeff_work pdurbin, rt is in fedora
18:12 SEJeff_work Should be very easy with mockchain to backport all of the deps if they aren't already in EPEL (they likely are)
18:17 pdurbin     hmmm, it is?
18:17 SEJeff_work If you manage RHEL and RHEL-ish systems, mock + mockchain are very good tools to know. Well provided you are comfortable with managing your own yum repos.
18:18 pdurbin     Fedora Package Database -- Invalid Package Name - https://admin.fedoraproject.org/pkgdb/acls/name/rt
18:18 SEJeff_work pdurbin, https://admin.fedoraproject.org/pkgdb/acls/name/rt3 :)
18:19 pdurbin     um. is anyone patching it?
18:19 SEJeff_work pdurbin, FYI: http://dl.fedoraproject.org/pub/epel/testing/6/x86_64/rt3-3.8.13-1.el6.1.noarch.rpm
18:20 pdurbin     Sat Jun 02 2012 Ralf Cors├ępius <corsepiu@fedoraproject.org> - 3.8.13-1 http://pkgs.fedoraproject.org/gitweb/?p=rt3.git;a=blob;f=rt3.spec;hb=HEAD
18:22 pdurbin     hmm. i was kind of hoping for RT 4.  but i guess that isn't very enterprisey of me :)
18:32 pdurbin     huh. Enoch Root. somehow i never noticed that before
19:13 tychoish    15:12 < evangoer> RT @Pinboard: Proud to reassure my users that Pinboard passwords are not only hashed and  salted, but pan-seared and dusted with a saffron quince reduction
19:15 pdurbin     heh
20:08 pdurbin     HTML9 Responsive Boilerstrap JS - http://html9responsiveboilerstrapjs.com
20:24 magoo       for those in here using aws, are you just assigning an elasticip to every node? I ask because I recently noticed that internal/external IPs change on reboots
20:25 pdurbin     um. we only have one host on aws and i always use a hostname to ssh to it, so i'm not sure
20:26 magoo       i haven't paid attention to it but since the hostname is created from the ip, I need to find out if the hostname also changes on rebott
20:36 shuff       back when i was using AWS we did not believe in reboots :)
20:36 shuff       we just clobbered the instance and provisioned another
20:44 magoo       shuff: i don't either but my concern is an instance going down and coming up with a different ip which means updating chef data to point everything correctly
20:44 magoo       maybe i'm overthinking this