Edit - Cane made one point already while I was typing, leaving my post as it was.
My girlfriend pointed out how silly it seems to spend money on dirt when it is everywhere outside for free. She has a point none of the plants that grow wild outside start off in potting soil.
I'm leaving here several possible replies to that, for her to take in consideration as well
1) Among the plants you're growing, few (or perhaps none of them) grow in the wild where you're living. The soil in their habitat and in your area can be very different. Of course this is only relevant if you're customizing your potting soils to your plants.
2) Particularly if you grow indoors, or if you're starting seeds, pathogens are a problem and chances are the dirt you picked outside will have plenty of them. Insect eggs, fungi and even other dormant seeds mixed in the dirt can be a problem. Good potting soils, and of course peat pellets, perlite, vermiculite, arlite, potting grit, etc. are sterile.
3) Plants growing in the wild have gone through a serious selection. A few bushes are the result of many seeds, and often you will have little amounts of seeds to start with. Germination rates are important for you, and controlling your substrate improves them.
4) Dirt from outside will often be very dense and deliver poor aeration to potted plants. This is not an issue where plants have all the room they want to grow and can easily move their roots towards the best direction, but when you have limited space, you need to optimize your resources.
Of course you can go the cheap route and recycle dirt from your back yard if it has a proper composition for your seeds, but in that case sterilization (say, in an oven tray) is a good idea, particularly if you're growing indoors and you're going to sow valuable seeds in there. Also, throw in some perlite or vermiculite for good draining.
I'm personally into making my own potting mixes, particularly for starting seeds or cuttings indoors, but recycling is always a good idea if you're doing it right. I mean, gardeners do use composting bins, right?
Saludos,
Mandrake