What are my cheaper heating options to get ready for Winter, I have a fireplace but not an insert?

I don't want to have high heating bills, I thought about using the fireplace but I don't have the stove that goes in it and people have told me that it doesn't do much good unless you have the heating stove in the fireplace.