Inhibitor of apoptosis (IAP) proteins are overexpressed in many cancers and have been implicated in tumor growth, pathogenesis, and resistance to chemo- or radiotherapy. On the basis of the NMR structure of a SMAC peptide complexed with the BIR3 domain of X-linked IAP (XIAP), a novel series of XIAP antagonists was discovered. The most potent compounds in this series bind to the baculovirus IAP repeat 3 (BIR3) domain of XIAP with single-digit nanomolar affinity and promote cell death in several human cancer cell lines. In a MDA-MB-231 breast cancer mouse xenograft model, these XIAP antagonists inhibited the growth of tumors. Close structural analogues that showed only weak binding to the XIAP-BIR3 domain were inactive in the cellular assays and showed only marginal in vivo activity. Our results are consistent with a mechanism in which ligands for the BIR3 domain of XIAP induce apoptosis by freeing up caspases. The present study validates the BIR3 domain of XIAP as a target and supports the use of small molecule XIAP antagonists as a potential therapy for cancers that overexpress XIAP.