Up at NEMBA Fest I took the Fatboy on about 1/3rd of the rides, every ride was great up there, the Fatboy was awesome on the Darling Hill trails and I really couldn't say that I was ever disappointed that I brought it on a ride. On one of the Fatboy rides we got over to the Burke side and coming down some of the slightly more technical trails I had to slow way down so the bounce of the tires didn't get to the point where I couldn't control the bike any more. It was fun, but on that same trail later in the weekend I was on my RIP and it was a lot more fun, I could go really fast and there was no issue at all.
I ride Crandall Park a lot because its close, the trails there are largely non-technical (with a few technical spots) so I have ridden the Fatboy there pretty exclusively. Yesterday I took the RIP9 out there just because it was the bike that happened to be on the car rack. It was fun with the RIP9, but the fat bike is way more fun to crank around the corners there - and there are a LOT of corners. I got a KOM on a climb on last night's ride, so I guess that answers the "is it slower" question, but more often than not I'm riding for fun, not speed.
I've got a suspension fork on order for the Fatboy, we'll see how that changes the formula.