The ellipticity of the Rayleigh wave at the surface depends on the seismic structure beneath and in the vicinity of the seismological station where it is measured. We derive here the expression and compute the 3-D kernels that describe this dependence with respect to S-wave velocity, P-wave velocity and density. Near-field terms as well as coupling to Love waves are included in the expressions. We show that the ellipticity kernels are the difference between the amplitude kernels of the radial and vertical components of motion. They show maximum values close to the station, but with a complex pattern, even when smoothing in a finite-frequency range is used to remove the oscillatory pattern present in mono-frequency kernels. In order to follow the usual data processing flow, we also compute and analyse the kernels of the ellipticity averaged over incoming wave backazimuth. The kernel with respect to P-wave velocity has the simplest lateral variation and is in good agreement with commonly used 1-D kernels. The kernels with respect to S-wave velocity and density are more complex and we have not been able to find a good correlation between the 3-D and 1-D kernels. Although it is clear that the ellipticity is mostly sensitive to the structure within half-a-wavelength of the station, the complexity of the kernels within this zone prevents simple approximations like a depth dependence times a lateral variation to be useful in the inversion of the ellipticity.