I'm pretty sure that it's all server-sided. I didn't have to download anything to play on a server that had it. When I said people could still play vanilla, I meant they could just not do anything in the SF stuff, I guess.
*cough* Let's not discuss Slimefun on a different plugin's topic.