In this paper we present optical transfer operator (OTO) constructed based on the consideration of the general boundary-value problem of the radiation transfer theory for the case of a plane layer with a reflecting bottom and finite sources of radiation. The kernel of the OTO is constructed of the influence functions (IF) identical to the point spread function or of the spatial frequency characteristics (SFCH), which coincide with the optical transfer function (OTF). The IF and SFCH are versatile linear transfer functions of the system "atmosphere (ocean, cloudiness, hydrom) - underlying surface" that are determined from a solution of the boundary-value problem of the theory of radiation transfer for a plane layer with nonreflecting boundaries irradiated with a source of like a cw laser beam. We have constructed an OTO for the case of horizontally inhomogeneous boundary with an anisotropic reflection when no splitting of spatial and angular variables is used in the scattering coefficient. Such an OTO has the most general form and is expressed in terms of IF and SFCH. We show in this paper that all other expressions for OTO are particular cases of the derived formula.