I'm afraid there's no way to automatically filter them out (I won't bore you with the details, but the route-planning software doesn't really work that way), but they're highlighted in green, so you can spot them and drag them away.

Unsurfaced is "anything that isn't tarmac or concrete". The route-planner prefers tarmac to gravel, gravel to dirt, and so on, so it'll generally try to keep you on good surfaces when possible. If you click on a section of your route on the map, you'll get a little pop-up window which can show you pictures (in the UK) and the surface type if known.

As ever, surface data is dependent on what the OpenStreetMappers have found - it's not recorded in all circumstances, which is where the photos come in useful!

(I should say, of course, that the idea behind showing them in green is that you can then drag the route away from them if you don't like them!)