I am not real familiar with ESXi, but, my understanding is that this is a Virtual Machine.
VMware vSphere, known in many circles as "ESXi", for the name of the underlying hypervisor architecture, is a bare-metal hypervisor that installs directly on top of your physical server and partitions it into multiple virtual machines
There is no underlying operating system,
An installation of Data Protector requires a OS to be running. I think that you need to install your agents on one of the underlying Virtual Machines. If the ESXi is, itself, emulating some other OS, then, I would need to know what that is
