It's hard to decsribe the sense of freedom you can feel while whizzing around on a bike.
Even where hills are involved, there is a real sense of achievement and once you get even a small increase in fitness (believe me, I'm no Pendleton!), you'll find almost any hill achievable if you get down into the lowest gear and pant and sweat until you slowly get there.
Just give yourself an hour or two and head out to some quiet roads and pedal.
It improves fitness quickly
You'll notice a difference in fitness levels within a couple of weeks if you can cycle a little twice or three times a week. And once it becomes a little easier, you'll be able to really enjoy it rather than just struggling through.
No major expenditure required.
I bought my bike off a friend for £50. It's heavy, not very fast and old. But it was a great bike originally, it is the right size for me and gets me around my commutes when time and weather and distances permit. And it'll get me round 50 mile weekend rides. Until I have somewhere to store a better bike safely, this one is fine for a novice.