Doubt that this will work in VTL Mode, it appears like a physical Library to DP.
You need to set up a StoreOnce Store and integrate it as StoreOnce Backup system into DataProtector.
With that you can use all Dedup-Methods Source-, Server- and Target-Sided Dedup.
Don't think you want to use the Software-Store (deduplication done by a Gateway Server) since you already have a Device that has the Dedup capability integrated.