One way that I've found to try and decide if the network is the bottleneck or not is to perform a backup simulation of the client but send the output to nul. This should give you an idea of the speed that a VBDA can pull the data off the disks. One thing to be aware of is caching on the disk array you're using for the source data so perform the VBDA test a few times until you get a consistent result. The command line syntax to run on your client is:
vbda -vol /<source disk> -trees "/<source directory"> -out nul -profile
eg, if you wanted to run the test on F:\Temp then the syntax would be:
vbda -vol /F -trees "/Temp" -out nul -profile
After this you can create a nul file device on the media server to try sending a backup to. This will receive the backup data over the network and just dump it away, eliminating the destination disks from the performance equation, allowing you to compare the results from the first test to the second.
Good luck!
John