A Randomized Phase 2 Study of Combination Atezolizumab and CDX-1127 (Varlilumab) With or Without Addition of Cobimetinib in Previously Treated Unresectable Biliary Tract Cancers

• Pathologically confirmed biliary tract cancer, having received at least 1 prior line of systemic therapy, and received no more than 2 prior lines of therapy in the metastatic setting (disease recurrence =\< 6 months from the last dose of perioperative therapy/day of surgery \[whichever is more recent\] in resected patients will be considered the first line of therapy)

⁃ Includes intrahepatic cholangiocarcinoma (IHC), extrahepatic cholangiocarcinoma (EHC), and gallbladder carcinoma (GBC), but not Ampulla of Vater cancers

• Patients must have measurable disease by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1

• Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of atezolizumab, cobimetinib, and CDX-1127 (varlilumab) in patients \< 18 years of age, children are excluded from this study

• Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 80%)

• Absolute neutrophil count \>= 1,500/mcL

• Hemoglobin \>= 9.0 g/dl

• Platelets \>= 100,000/mcL

• Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (Patients with known Gilbert disease who have serum bilirubin level =\< 3 x ULN may be enrolled)

• Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 3 x institutional ULN

• Serum creatinine =\< 1.5 x institutional ULN OR

• Creatinine clearance \> 30 mL/min/1.73 m\^2 (calculated by Cockcroft-Gault method) for patients with creatinine levels above institutional normal

• Albumin \>= 3.0 g/dL

• Prothrombin time (PT)/activated partial thromboplastin time (aPTT) =\< 1.5 x ULN (This applies only to patients who do not receive therapeutic anticoagulation; patients receiving therapeutic anticoagulation, such as low-molecular-weight heparin or warfarin, should be on a stable dose)

• Creatine kinase (CK)/creatine phosphokinase (CPK) \< 5 x ULN

• Oxygen saturation \>= 92% on room air

• Left ventricular ejection fraction \> 50%

• Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial

• For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated

• Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load

• Patients must be willing to undergo 2 sets of core needle biopsies. If possible, biopsied sites should be different than those used for measurable disease/RECIST measurements, but this is not mandatory

• Patients must have an estimated life expectancy of greater than 3 months

• Patients must be able to swallow pills

• Patients should not have evidence of retinal pathology on ophthalmologic examination; or neurosensory retinal detachment, retinal vein occlusion (RVO), or neovascular macular degeneration

• The effects of atezolizumab, cobimetinib, and CDX-1127 (varlilumab) on the developing human fetus are unknown. For this reason, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and 5 months after the last dose of atezolizumab. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 5 months (150 days) after completion of atezolizumab, cobimetinib, and CDX-1127 (varlilumab) administration

• Ability to understand and the willingness to sign a written informed consent document