Mammalian nonsense-mediated mRNA decay (NMD) is a eukaryotic surveillance mechanism that degrades mRNAs containing premature translation termination codons. Phosphorylation of the essential NMD effector UPF1 by the phosphoinositide-3-kinase-like kinase (PIKK) SMG-1 is a key step in NMD and occurs when SMG-1, its two regulatory factors SMG-8 and SMG-9, and UPF1 form a complex at a terminating ribosome. Electron cryo-microscopy of the SMG-1-8-9-UPF1 complex shows the head and arm architecture characteristic of PIKKs and reveals different states of UPF1 docking. UPF1 is recruited to the SMG-1 kinase domain and C-terminal insertion domain, inducing an opening of the head domain that provides access to the active site. SMG-8 and SMG-9 interact with the SMG-1 C-insertion and promote high-affinity UPF1 binding to SMG-1-8-9, as well as decelerated SMG-1 kinase activity and enhanced stringency of phosphorylation site selection. The presence of UPF2 destabilizes the SMG-1-8-9-UPF1 complex leading to substrate release. Our results suggest an intricate molecular network of SMG-8, SMG-9 and the SMG-1 C-insertion domain that governs UPF1 substrate recruitment and phosphorylation by SMG-1 kinase, an event that is central to trigger mRNA decay.,© The Author(s) 2015. Published by Oxford University Press on behalf of Nucleic Acids Research.