The membrane trafficking of cation-independent mannose 6-phosphate receptor (CI-M6PR) between the trans-Golgi network (TGN) and endosomal compartments is not only critical for maintaining lysosomal function but also a well-known event for understanding molecular and cellular mechanisms in retrograde endosome-to-TGN trafficking. Although it has been well established in literature that the C-terminus of bovine CI-M6PR determines its retrograde trafficking, it remains unclear whether the luminal domain of the protein plays a role on these sorting events. In this study, we found that partial deletion of luminal domain of human CI-M6PR mistargeted the mutant protein to non-TGN compartments. Moreover, replacing the luminal domain of both bovine and human CI-M6PR with that from irrelevant membrane proteins such as CD8 or Tac also altered the TGN targeting of the chimeric proteins. On the other hand, only short sequence from HA fused with the transmembrane domain and C-terminus of the receptor, HA-hCI-M6PR-tail, resulted in its preferential targeting to TGN as for the full length receptor, strongly suggesting that sorting of the receptor may be influenced by luminal sequence. Furthermore, using this luminal truncated form of HA-hCI-M6PR as a model cargo, we found that the trafficking of the chimeric protein was regulated by the retromer complex through interacting with SNX5. In conclusion, our study strongly suggested that the disrupted luminal domain from hCI-M6PR or other irrelevant membrane proteins interfere with the process of membrane trafficking and TGN targeting of CI-M6PR.