Outside London, most of the Thames Path isn’t cyclable: the surfaces are too rough, the path too narrow. But you can get a good taste of the Thames by following NCN route 5 south from Oxford to Abingdon. Rather than taking the main route through Grandpont to begin with, follow the riverside path south until they rejoin at the bypass bridge.
From there, it’s quiet lanes and cycle paths all the way to Abingdon – just 8 miles in all. Abingdon no longer has a railway station for return transport, but Radley station is en route.