This study is researching an investigational drug called REGN5668. Participants will receive additional investigational drugs in combination with REGN5668. These additional drugs include cemiplimab or REGN4018 (with or without sarilumab). The main purposes of this study are to: * Learn about the safety and profile of any side effects from the study drugs and to determine the highest, safe dose that can be given to participants with ovarian cancer or cancer of the uterus * Look for signs that the study drugs can treat ovarian cancer or cancer of the uterus This study has 2 parts. The purpose of Part 1 (Escalation) to find the highest, safe dose of the study drug(s). The purpose of Part 2 (Expansion) is to use the doses chosen in Part 1. Participants with cancer of the uterus will only participate in Part 2. The study is looking at several other research questions, including: * Side effects that may be experienced by participants taking REGN5668 alone and/or in combination with cemiplimab or REGN4018 * How REGN5668 works in the body either alone and/or in combination with cemiplimab or REGN4018 * How much of the study drugs (REGN5668, cemiplimab, REGN4018) are in the blood * To see if REGN5668 in combination with cemiplimab or REGN4018 works to treat cancer * To find out how safe, tolerable, and effective in mitigating Cytokine Release Syndrome (CRS) sarilumab pretreatment is when given before REGN4018