To troubleshoot the problem you could make a new backup spec, of the filesystem (selecting a similar amount of data) of the problem system, and keep everything else in the spec as similar as possible to the Oracle backup spec.
Try running that, is it slow or fast ?