The conserved oligomeric Golgi (COG) complex is a peripheral membrane protein complex which orchestrates tethering of intra-Golgi vesicles. We found that COG1-4 (lobe A) and 5-8 (lobe B) protein assemblies are present as independent sub-complexes on cell membranes. Super-resolution microscopy demonstrates that COG sub-complexes are spatially separated on the Golgi with lobe A preferential localization on Golgi stacks and the presence of lobe B on vesicle-like structures， where it physically interacts with v-SNARE GS15. The localization and specific interaction of the COG sub-complexes with the components of vesicle tethering/fusion machinery suggests their different roles in the vesicle tethering cycle. We propose and test a novel model that employs association/disassociation of COG sub-complexes as a mechanism that directs vesicle tethering at Golgi membranes. We demonstrate that defective COG assembly or restriction of tethering complex disassembly by a covalent COG1-COG8 linkage is inhibitory to COG complex activity， supporting the model.