The transformation of light carrying spin angular momentum (SAM) into optical field vortices carrying orbital angular momentum (OAM) has been of great interest in recent years. The interaction between two optical fields, each carrying one of these degrees of freedom, and the transfer of the resulting angular momentum product to matter is rarely discussed. Here we measure the interaction between 3D light carrying axial SAM and 2D plasmon-polariton vortices carrying high-order transverse OAM. The interaction is mediated by two-photon absorption within a gold surface, and the resulting angular momentum mixing is imprinted into matter by excitation of electrons photo-emitted into the vacuum. In our approach, we follow the spatio-temporal dynamics of the plasmonic fields using time-resolved photoemission electron microscopy (TR-PEEM) with sub-femtosecond temporal resolution.