Generally speaking, perormance problems are notoriously hard to diagnose
I am attaching a document which may provide some help
From your description, you are backing up to a file library. Are you only using 2 writers?. You should think about having one writer per object being backed up
File libraries and VTLs need to run with a concurrency of '1', while with tape devices, you normally want to set concurrency to 10 or 15, if you are backing up enough objects, to enable drive streaming
I am a little confused by your backup environment description, so, let me ask a basic question. Are you backing up over the LAN, or is your File library considered local. AN easy way to check this is to see if the hostname referenced in a VBDA sesion message is the same as that referenced in BMA messages. If they are not the same, then you are backing up over teh LAN, and probably looking at a network bottleneck