Time  Nick         Message
11:25 pdurbin      agoddard: i posted your iSCSI + libvirt thing: [CentOS-virt] Basic shared storage + KVM - http://lists.centos.org/pipermail/centos-virt/2012-June/002946.html
12:43 pdurbin_m    half of stock trades on rhel
12:49 SEJeff       ?
12:56 pdurbin_m    according to paul cormier at the red hat summit. he's wearing red hat cuff links :)
12:58 pdurbin_m    he's talking all about openshift
13:01 SEJeff       Stock trading and virt do *not* go together. I can speak on that as someone who works for a large "designated market maker" wall st firm
13:01 SEJeff       However that statement for the most part is true. A majority of banks and exchanges are all infiniband + RHEL
13:02 SEJeff       Some freebsd, some solaris, no one of any significance uses windows to trade. The latency just isn't there.
13:04 pdurbin_m    yeah, he did not argue for virt + trading :)
13:04 SEJeff       Oh I thought all of the open* from redhat was virt. Forgive my ignorance then
13:05 pdurbin_m    he does seem to argue you should ditch netapp and emc and run linux servers for storage
13:05 SEJeff       Thats actually a shock
13:06 SEJeff       With no filesystem supporting online grow/shrink and any decent block level replication solution at least
13:06 pdurbin_m    whorka ran/runs netapp so well I'm a big fan
13:06 SEJeff       Yeah, I run netapps as well
13:06 SEJeff       Big fan
13:07 SEJeff       The only real downside of netapps is that they are cpu based (unlike a Bluearc)
13:07 SEJeff       So when you hit the performance peak, performance will quickly go through the floor and it is awful
13:07 pdurbin_m    CPU based?
13:07 SEJeff       but WAFL is incredibly convenient and there isn't much of anything in the industry that is better than snapmirror, netapp's replication technology
13:08 SEJeff       pdurbin_m, How familar are you with a netapp?
13:08 pdurbin_m    meh. kinda
13:08 SEJeff       OnTap (netapp's prop OS) is a BSD based derivative. Their only serious claim to fame is a filesystem they dub WAFL and a raid 6-ish derivative they call RAID-DP
13:08 pdurbin_m    I've heard of wafl :)
13:09 SEJeff       The magic all happens in software, so all of the filesystem stuff and all of the deduplication or whatnot happens on the systems cpu
13:09 SEJeff       A low end netapp runs a celeron proc
13:09 SEJeff       Here let me see what one of our beefier ones runs
13:10 SEJeff       pdurbin_m, On a bit beefier filer: http://hastebin.com/dotiyajoba.coffee
13:10 pdurbin_m    what is there besides CPU based?
13:10 SEJeff       ASIC based
13:10 SEJeff       So with a netapp, when you hit the top end of the performance curve, the cpu can't keep up
13:10 pdurbin_m    ah
13:11 SEJeff       Since the magic in a netapp is all in the filesystem... If the cpu can't keep up, the performance will actually drop through the floor and you are stuck with a real mess for awhile until the IO slows down
13:11 SEJeff       pdurbin_m, With a bluearc, that aspect of things is all ASIC based and performance is a (very high) straight line that is steady up until the maximum hardware limit.
13:12 pdurbin_m    cool. I hear we used to have a bluearc
13:13 pdurbin_m    we're mostly an emc shop. and build it yourself
13:13 SEJeff       pdurbin_m, NetApp gave themselves a bit of breathing room when they bought Spinnacer and integrated it all into OnTap 8. Spinnacer was the only vendor to truly solve the active/active clustered storage problem, so you can chain up to 12 beefy netapp filers (with ontap 8) and get the almost combined throughput of all of them together
13:13 SEJeff       Yeah we have a few mill worth of emc. Can't stand emc myself. Overpriced garbage
13:14 SEJeff       but bluearc uses ndmp as their "replication" tech. For that use case, ndmp is complete crap
13:15 SEJeff       Linux had one thing that was vaguely close to competing with netapp, http://zumastor.org, but that project died when google fired the lead developer and our company hired (then fired) him as well.
13:15 pdurbin_m    whorka set up good replication and disaster recovery for us. snapmirror and such
13:15 pdurbin_m    heh
13:15 SEJeff       Yes snapmirror is the best. It is trivial to quiesce/break a snapmirror, flip the relationship, and then start it back when performing an entire datacenter failover. Ticketmaster bet their entire DR strategy around that sadly unique feature that only netapp has
13:16 pdurbin_m    a couple times we flipped over to our other data center
13:17 pdurbin_m    sap guy talking now. do I want sap?
13:17 SEJeff       pdurbin_m, Now if you were using anything such as a bluearc which uses ndmp for that... When you flip the relationship, it requires a full resync over the wan of 100% of the data vs netapp which just copies over the incremental snapshot differences
13:17 SEJeff       I dunno
13:18 pdurbin_m    I don't think so :)
13:18 SEJeff       and 100% of the data is re-re-copied when you fail back and reverse the replication direction yet again
13:18 SEJeff       ha
13:19 pdurbin_m    yeah, that bluearc replication sounds not so good
13:20 SEJeff       Right. I have a feeling that netapp has a patent or something on the way they do replication. Sadly, it is the most obvious way to do it properly
13:22 pdurbin_m    death to software patents
13:26 SEJeff       here here
13:28 pdurbin_m    dr. richard hackathorn? srsly? great name :)
13:31 SEJeff       win
13:31 SEJeff       Hackathorn's Hackathon
13:41 pdurbin_m    heh, "quants" (sp?). I first hear this term from my brother, who works for goldman
13:41 SEJeff       Not all HFT is quant
13:41 SEJeff       Our company is strictly arb and the variants thereof
13:42 SEJeff       market making is good for the markets as a whole
13:42 SEJeff       quant can cause things like the flash crash
13:42 pdurbin_m    crimsonfubot: google hft
13:42 crimsonfubot pdurbin_m: High-frequency trading - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/High-frequency_trading>; Harbor Freight Tools – Quality Tools at Discount Prices Since 1977: <http://www.harborfreight.com/>; Houston Federation of Teachers - Home: <http://tx.aft.org/hft/>; Hartford Federation of Teachers - Home: <http://www.hft-1018.org/>; High-Frequency Trading (HFT): (1 more message)
13:42 pdurbin_m    right right
13:43 pdurbin_m    arb?
13:44 SEJeff       crimsonfubot, google wikipedia arbitrage
13:44 crimsonfubot SEJeff: Arbitrage - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/Arbitrage>; Arbitrage betting - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/Arbitrage_betting>; Arbitrage (film) - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/Arbitrage_(film)>; Risk arbitrage - Wikipedia, the free encyclopedia: <http://en.wikipedia.org/wiki/Risk_arbitrage>; (2 more messages)
13:44 SEJeff       nice!
13:44 pdurbin_m    :)
13:44 SEJeff       in simple terms, it is the possibility of a risk-free profit at zero cost.
13:44 SEJeff       arb is the only guaranteed way to make money in the markets with 1 caveaut
13:45 SEJeff       You have to be the fastest in the specific market you are in to make money doing it. If you are a "market maker" you often make new markets that no one else is trading in, so... you convince other people to join and you make money
13:45 SEJeff       very very rough not so great description ^^
13:46 pdurbin_m    accenture guy up now. they have 40,000 java devs
13:47 pdurbin_m    I'll run it past my brother :)
13:48 SEJeff       pdurbin_m, Ask him about market makers / designated mms and how them + their strats fit into the market.
13:51 pdurbin_m    I sent him a link :)
13:53 pdurbin_m    red hat ball caps on every seat in here. hundreds of them
13:55 pdurbin_m    picked up a pint glass last night: https://mobile.twitter.com/philipdurbin/status/217770666420748290
13:56 SEJeff       Ha
13:56 SEJeff       check your pm as well
14:49 pdurbin_m    crimsonfubot: google glusterfs swift ufo
14:49 crimsonfubot pdurbin_m: Fedora 17, OpenStack Essex & Gluster 3.3: All Smushed Together ...: <http://blog.jebpages.com/archives/fedora-17-openstack-and-gluster-3-3/>; Chapter 13. Managing Unified File and Object Storage: <https://docs.redhat.com/docs/en-US/Red_Hat_Storage/2/html/User_Guide/chap-User_Guide-UFO.html>; Index of /pub/gluster/glusterfs/LATEST/UFO: (2 more messages)
15:22 pdurbin_m    gluster chooses consistency over performance
15:31 pdurbin_m    iPhone battery 45% :(
17:57 pdurbin_m    live migration of virtual machine disk between storage domains... I'm at a RHEV 3.1 talk
18:01 pdurbin_m    some kind of gluster support as a tech preview
18:03 pdurbin_m    funny that vm snapshots are a new feature. we're doing this with qcow2 disk images with basic libvirt
18:04 pdurbin_m    RHEV is definitely its own thing. I'm much more used to basic libvirt/KVM
18:05 pdurbin_m    the main thing is the GUI, of course. and high availability, which we don't have
18:06 pdurbin_m    nice that with RHEV there seems to be feature parity across the GUI, the cli, and the rest API
18:09 pdurbin_m    interesting that by a show of hands 80% or more in this room are running VMware. maybe 120 people
18:10 pdurbin_m    room 105 RHEV campground, so I don't forget :)
18:35 pdurbin_m    was one of two people to boo at hearing "you don't need perl" :)
18:36 pdurbin_m    guy was talking about RHEV vs KVM
18:36 sjoeboo_     nice perl joke
18:36 pdurbin_m    how you have to do more heavy lifting yourself with just KVM
18:37 sjoeboo_     scripts...? which i don't mind
18:37 sjoeboo_     then yo uknow whats going on..
18:37 pdurbin_m    bummer the distributed file system talk filled up
18:37 pdurbin_m    the guy behind me is asleep :)
18:38 sjoeboo_     yeah, a few 100 people in front of me even! so much for pre-registering being worth anything
18:38 pdurbin_m    turned around to look at his name tag and he woke up :)
18:39 sjoeboo_     haha
18:39 sjoeboo_     report him!
18:39 pdurbin_m    lol
18:40 sjoeboo_     these reports sound like ganglia
18:40 sjoeboo_     only via the hypervisor
18:40 sjoeboo_     neat?
18:40 pdurbin_m    in the last RHEV talk you could see they're using iSCSI
18:40 sjoeboo_     interesting
18:40 pdurbin_m    I'm pro reports
18:41 sjoeboo_     i can't can't see the backend managinf of volumes being awesome..maybe they have more secret sauce/better iscsi deployments
18:41 sjoeboo_     yeah yeah, reports ftw
18:41 pdurbin_m    they also made noise about better gluster support coming in future versions of RHEV
18:42 sjoeboo_     nice, yeah, i mean, out EL6 kvm hosts are keen to talk ot gluster as a pool type
18:44 sjoeboo_     a unified interface w/o X11 to livirt would be great like this...
18:44 * sjoeboo_   searches for centos 6 ovirt pkgs
18:46 pdurbin_m    yes. we are kind of cobbling together our own tools...  and we have a heavy dependence on X11
18:47 sjoeboo_     i don't mind, but, i know others do
18:47 sjoeboo_     being open and all..
18:47 pdurbin_m    but promise we we'll buy RHEV before we write some webapp on top of jboss! :)
18:48 pdurbin_m    isn't it hilarious that live snapshots are a new feature?
18:48 sjoeboo_     yeah yeah
18:48 pdurbin_m    imagine! live! no need to shut down the vm! it's the future! ;)
18:48 sjoeboo_     no, we can get ovirt up
18:48 sjoeboo_     might take some effort
18:49 sjoeboo_     but it can be done
18:50 pdurbin_m    the RHEV guys are more bullish on putting kvm disk images on gluster than the gluster guys are :)
18:51 sjoeboo_     yeah yeah
18:53 pdurbin_m    got to ask my question about configuring gluster with replication or not
18:53 sjoeboo_     obvioulsy storage for this is a big deal
18:53 sjoeboo_     we're not alone
18:54 sjoeboo_     ooo, hotplug
18:54 pdurbin_m    gluster guys don't recommend replication for this use case, right? KVM images on gluster...
18:54 sjoeboo_     i wa suprised that DIDNT work
18:54 sjoeboo_     they didn't recommend it..
18:54 sjoeboo_     i think its dependant on the vm
18:54 sjoeboo_     like, website, prob okay
18:54 sjoeboo_     eol-app servers doing lots of writes...not so much
18:55 sjoeboo_     and all the issues seemed around recovery from loosing a brick
18:55 pdurbin_m    agoddard: ^^ :)
18:55 sjoeboo_     not "normal" operation
18:55 sjoeboo_     ha
18:56 pdurbin_m    I mean, you want replication, right? I would think so
18:56 sjoeboo_     i do
18:56 sjoeboo_     yeah
18:56 sjoeboo_     well...it would be nice
18:56 pdurbin_m    sure, set up raid too but redundancy at many levels
18:56 sjoeboo_     yeah yeah, all gluster we do is ontop of raid 5 or 6 already
18:57 pdurbin_m    as we talked about, we can set up a storage pool backed by gluster and have a bake off vs nfs :)
18:57 sjoeboo_     yep
18:57 sjoeboo_     thats the idea with the m3000i's + gfs01-06
18:57 sjoeboo_     just need to get vms off vmware THEN 04-06 first
18:58 pdurbin_m    this direct LUN stuff is what agoddard wants. for performance
18:58 sjoeboo_     yeah
18:58 sjoeboo_     they can do that now
18:58 sjoeboo_     we just don't have build scripts etc for it currently
18:58 sjoeboo_     or storage for them
18:59 pdurbin_m    true
18:59 sjoeboo_     well, maybe we do, i don't know the grounds of whatever agreement/relationship there is in terms of whats free/etc
19:00 sjoeboo_     live snaps are new cause its from upstream kvm, which, is what we run :-)
19:02 pdurbin_m    heh. right
19:02 pdurbin_m    RHEV guys talking about live storage migration
19:03 sjoeboo_     mm
19:03 sjoeboo_     we could use that...now...but this would require all our stuff already being RHEV :-(
19:04 pdurbin_m    tweeting to the gluster guys we've met, linking back here: https://mobile.twitter.com/philipdurbin/status/218056394262319104
19:05 pdurbin_m    yeah, live storage migration would be great
19:06 pdurbin_m    sometimes I feel like RHEV (or ovirt) is more what we want than openstack
19:06 sjoeboo_     yep
19:06 sjoeboo_     i'm onboard with that
19:06 pdurbin_m    westmaas: sorry :)
19:07 sjoeboo_     i mean, yes, the ability of our users to fire up a vm/instance when they want would be nice
19:07 pdurbin_m    but maybe that's just because we're coming from a VMware background
19:07 sjoeboo_     but this sounds nicer to me!
19:07 sjoeboo_     yeah, speak for yourself!
19:08 pdurbin_m    openstack is more of an alternative to amazon aws than VMware
19:08 sjoeboo_     exactly
19:08 pdurbin_m    with all the i
19:08 pdurbin_m    with all the images and what not
19:09 sjoeboo_     mmm
19:10 sjoeboo_     the port mirroring for IDS is neato, not sure we'd use that...maybe for hrci stuff, but neat nevertheless
19:11 pdurbin_m    yeah
19:13 pdurbin_m    crimsonfubot: google vdsm
19:13 crimsonfubot pdurbin_m: Category:Vdsm - oVirtWiki: <http://ovirt.org/wiki/Category:Vdsm>; VDSM Overview: <http://www.ovirt.org/wp-content/uploads/2011/11/ovirt-vdsm.pdf>; Vdsm - Virtual Desktop and Server Management Daemon KVM ...: <http://www.linux-kvm.org/wiki/images/6/66/Vdsm.pp.pdf>; VDSM - Acronyms - The Free Dictionary: <http://acronyms.thefreedictionary.com/VDSM>; vdsm-devel Info Page: (1 more message)
19:15 pdurbin_m    this hook thing is pretty cool. allows for some flexibility
19:15 sjoeboo_     yeah yeah
19:15 sjoeboo_     snap whenever a shutdown is sent
19:15 sjoeboo_     etc
19:16 sjoeboo_     (just snap, not standalone disk image)
19:16 pdurbin_m    that's the thing with basic libvirt. we can do whatever we want
19:16 sjoeboo_     sometimes i think doing lots of the quick, tiny in-image snapshots might be better than doing all the disk-splits...about 1000% less io heavy
19:16 pdurbin_m    enough rope to hang ourselves :)
19:16 sjoeboo_     yep!
19:17 pdurbin_m    hmm, yeah, maybe
19:18 pdurbin_m    need this quota stuff. and the ability for users to spin up their own VMs ...
19:18 sjoeboo_     yeah, the thin provisioning stuff is shady
19:18 sjoeboo_     oversubscribe stuff quickly
19:20 sjoeboo_     yes yes no fibre channel
19:20 sjoeboo_     ha
19:20 pdurbin_m    hee hee
19:21 sjoeboo_     9 minutes of battery left, made it
19:22 pdurbin_m    I guess I want to ask... does RHEV give me more or less flexibility with regard to storage vs basic libvirt...
19:22 pdurbin_m    3% battery left on this iPhone :(
19:22 sjoeboo_     more managment/oversight i think would be the answer
19:22 pdurbin_m    I've added enough noise to the Internet today :)
19:23 sjoeboo_     i think the libvirt felxability would still be there, this would all just sit on top to make a bunch of things easier/point+click
19:23 sjoeboo_     okay, talk over
19:25 westmaas     pdurbin_m: some day
19:25 westmaas     some day you are gonna be so sad you have to spin up all the instances yourself
19:25 westmaas     !