We are going through a paradigm shift in how autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs) are deployed in marine robotics applications such as subsea inspection, monitoring, and surveillance. Despite significant developments across academia and industry in recent years, there remain many challenges and important research frontiers. This talk will highlight these challenging open problems and evolving technologies that attempt to solve these by vision, language, haptics, optics, and acoustic modalities. I will present some of our recent findings and advancements made in: (1) interactive subsea mission planning for AUVs, (2) human-machine shared autonomy for ROVs, and (3) low-power standalone operation or subsea sensors and surveillance agents.