That's a fair question. I would say it really depends on what kind of used bike you find. For instance, if you buy an old, shitty Huffy or some other box-store brand, it would be better to recycle it, as getting it back into useable spec will take a lot of parts, time, and effort. It would probably be better to just buy a new bike that takes less labor and parts to make serviceable.
If you get something nicer, for instance an older (70s) Schwinn, or a steel frame anything, etc, it is probably worth it to bring it back to life.