Automation is going to be the key to success. Your Smelter can interface with adjacent Hoppers of any type.
However, if you're not specific with your instructions, then all sorts of bad things could happen! Set the permissions now, to have one Add-Only hopper and one Remove-Only hopper. Convey Ore into the Remove-Only Hopper and Ingots will be added to the Add-Only Hopper. Remove from the Add-Only Hopper via Conveyors.