Tag Archives: Activity Recognition

Robot Assisted Laparoscopic Hysterectomy

Sample phase segmentation of hysterectomy by a human annotator

Aims: Develop methods that can automate segmentation of procedure into phases and assessment of technical skill

Description: This project aims to develop methods that can automatically 1) segment the surgical procedure into constituent steps (phases), and 2) assess the surgical skill within each step. We have collected a dataset of 30 robot-assisted laparoscopic hysterectomy procedures performed using the da Vinci system. This contains endoscopic stereo video, instrument and console motion, and user events on the system.

Funding: Link Foundation Fellowship, NSF NRI Award 1227277

People: Anand Malpani, S. Swaroop Vedula, Chi Chiung Grace Chen, Gregory D. Hager

Publications:

  1. Berges AJ, Vedula SS, Chen CCG, Malpani A. Is There a Relationship Between Warm-Up Virtual Reality Simulation and Trainee Robot-Assisted Laparoscopic Hysterectomy Performance? Journal of Minimally Invasive Gynecology. 2018 Nov 1;25(7):S92. (under preparation)
  2. Malpani A, Martinez N, Vedula SS, Hager GD, Chen CCG. Automated Skill Classification using Time and Motion Efficiency Metrics in Vaginal Cuff Closure. Society of Gynecologic Surgeons, 44th Annual Scientific Meeting. Orlando, FL; 2018. (under revision)
  3. Malpani A, Arora S, Vedula SS, Chen CCG, Hager GD. Crowdsourcing surgical activity summaries for phase recognition. LABELS: International Workshop Large-Scale Annotation of Biomedical Data and Expert Labels Synthesis. Granada, Spain; 2018.
  4. Malpani A, Lea C, Chen CCG, Hager GD. System events: readily accessible features for surgical phase detection. Int J CARS. 2016 May 13;11(6):1201–1209.
  5. Chen CCG, Tanner E, Malpani A, Vedula SS, Fader AN, Scheib SA, Green IC, Hager GD. Warm-up before robotic hysterectomy does not improve trainee operative performance: a randomized trial. J Minim Invasive Gynecol. 2015 Nov;22(6, Supplement):S34.

Cataract Surgery

Aims: Develop technology to enable an automated co-pilot for eye surgeons.

Summary: This project aims to develop technology to enable a system that can function as an automated co-pilot for surgeons using cataract surgery as a testbed procedure. Given video of the surgical field during a procedure, this system segments the procedure into constituent activities, assign a skill rating for each activity, and provide commensurate feedback to the surgeon. This project relies upon multiple modes of data including video images and verbal and textual surgical narratives.

Funding: The Wilmer Eye Institute Pooled Professors Fund 2016 (PI: Dr. Shameema Sikder), and an unrestricted research grant to The Wilmer Eye from Research to Prevent Blindness

People: Shameema Sikder, S. Swaroop Vedula, Gregory D. Hager, Anand Malpani, Sidra Zafar, Tae Soo Kim

Publications:

  1. Kim TS, O’Brien M, Zafar S, Hager GD, Sikder S, Vedula SS. Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery. Int J CARS [Internet]. 2019 Apr 11 [cited 2019 Apr 19]; Available from: https://doi.org/10.1007/s11548-019-01956-8
  2. Yu F, Silva Croso G, Kim TS, Song Z, Parker F, Hager GD, Reiter A, Vedula SS, Ali H, Sikder S. Assessment of Automated Identification of Phases in Videos of Cataract Surgery Using Machine Learning and Deep Learning Techniques. JAMA Netw Open [Internet]. 2019 Apr 5 [cited 2019 Dec 17];2(4). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6450320/ PMCID: PMC6450320
  3. Kim TS, Malpani A, Reiter A, Hager GD, Sikder S, Swaroop Vedula S. Crowdsourcing Annotation of Surgical Instruments in Videos of Cataract Surgery. In: Stoyanov D, Taylor Z, Balocco S, Sznitman R, Martel A, Maier-Hein L, Duong L, Zahnd G, Demirci S, Albarqouni S, Lee S-L, Moriconi S, Cheplygina V, Mateus D, Trucco E, Granger E, Jannin P, editors. Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. Granada, Spain: Springer International Publishing; 2018. p. 121–130.