Saelyne Yang (양세린)

Postdoctoral Fellow @ CMU

Recent News
  • 🛫 (Jun. 2026) Attended my first in-person PC meeting for UIST 2026 in Ann Arbor!
  • 🏆 (Jun. 2026) GUIDE won an Outstanding Paper Award at the 2nd Workshop on MMRAgI at CVPR 2026!
  • 🛫 (Jun. 2026) Attending CVPR 2026 to present GUIDE as a full paper and to give an oral presentation at a workshop!
  • 👩‍💼 (Mar. 2026) Started as a Postdoctoral Fellow at CMU!
  • 📝 (Feb. 2025) 1 paper accepted to CVPR, 1 paper accepted to CHI!
  • 👩‍💼 (Feb. 2026) Serving as Program Committee for UIST 2026.
  • 🎓 (Dec. 2025) Successfully defended my Ph.D. Thesis!
  • 👩‍💼 (Jul. 2025) Serving as Program Committee for AAAI 2026.
  • 📝 (Jan. 2025) My Adobe internship paper has been accepted to IUI 2025!
  • 🏆 (Nov. 2024) Received a Special Recognition for Outstanding Reviews at CHI 2025.
  • 🛫 (Oct. 2024) Attended UIST 2024 in Pittsburgh!🇺🇸
  • 📝 (Aug. 2024) Doctoral Consortium and a paper have been accepted to UIST 2024!
  • 🛫 (May. 2024) Attended CVPR 2024 and presented workshop papers!🇺🇸
  • 👩‍💼 (May. 2024) Started a summer internship at Adobe Seattle!
  • 🛫 (May. 2024) Attended CHI 2024 in Hawaii!🇺🇸
  • 🛫 (Mar. 2024) Attended IUI 2024 in Greenville!🇺🇸
  • 🛫 (Feb. 2024) Attended AAAI 2024 in Vancouver!🇨🇦

Hi! I'm a Postdoctoral Fellow at Carnegie Mellon University's Human-Computer Interaction Institute (HCII), working with Jeff Bigham. I earned my Ph.D. from KAIST, advised by Juho Kim, with support from the NAVER PhD Fellowship and the Kwanjeong Scholarship. I've had the opportunity to intern at Adobe Research, Autodesk Research, and LG AI Research.

My research focuses on helping people learn new skills and perform complex tasks. I have developed AI-based systems that use instructional videos and AI agents to support users in completing procedural tasks. My current research focuses on building user-aware and collaborative AI agents that can adapt to individual users and work alongside them.
Research Interests Human-AI Interaction Human-Agent Collaboration GUI Agents Multimodal Understanding Procedural Task Learning Skill Learning

Publications

Contexty: Capturing and Organizing In-situ Thoughts for Context-Aware AI Support

Yoonsu Kim, Chanbin Park, Kihoon Son, Saelyne Yang, Juho Kim

UIST 2026
pdf
GUIDE: A Benchmark for Understanding and Assisting Users in Open-Ended GUI Tasks

Saelyne Yang, Jaesang Yu, Yi-Hao Peng, Kevin Qinghong Lin, Jae Won Cho, Yale Song, Juho Kim

CVPR 2026
🏆 Outstanding Paper Award at the 2nd Workshop on MMRAgI at CVPR 2026
pdf Project Website
"I Can't Keep Up": Accessibility Barriers in Video-Based Learning for Individuals with Borderline Intellectual Functioning

Hyehyun Chu, Seungju Kim, Chen Zhou, Yu-Kai Hung, Saelyne Yang, Hyun W. Ka, Juho Kim

CHI 2026
pdf
VideoMix: Aggregating How-To Videos for Task-Oriented Learning

Saelyne Yang, Anh Truong, Juho Kim, Dingzeyu Li

IUI 2025
pdf
Silent Impact: Tracking Tennis Shots from the Passive Arm

Junyong Park, Saelyne Yang, Sungho Jo

UIST 2024
pdf
AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

Saelyne Yang, Jo Vermeulen, George Fitzmaurice, Justin Matejka

CHI 2024
pdf
ExpressEdit: Video Editing with Natural Language and Sketching

Bekzat Tilekbay, Saelyne Yang, Michal A Lewkowicz, Alex Suryapranata, Juho Kim

IUI 2024
Also presented at the HAI-GEN Workshop at IUI 2024
pdf Project Website
YTCommentQA: Video Question Answerability in Instructional Videos

Saelyne Yang, Sunghyun Park, Yunseok Jang, Moontae Lee

AAAI 2024
Also presented at the LPVL Workshop at CVPR 2024
pdf Poster Dataset
Towards Content-based Pixel Retrieval in Revisited Oxford and Paris

Guoyuan An, Woo Jae Kim, Saelyne Yang, Rong Li, Yuchi Huo, Sungeui Yoon

ICCV 2023
pdf Dataset
Beyond Instructions: A Taxonomy of Information Types in How-to Videos

Saelyne Yang, Sangkyung Kwak﹡, Juhoon Lee﹡, Juho Kim

CHI 2023
pdf Project Website Talk
AVscript: Accessible Video Editing with Audio-Visual Scripts

Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang ‘Anthony’ Chen, Young-Ho Kim, Amy Pavel

CHI 2023
pdf
CatchLive: Real-time Summarization of Live Streams with Stream Content and Interaction Data

Saelyne Yang, Jisu Yim, Juho Kim, Hijung Valentina Shin

CHI 2022
pdf Project Website
SoftVideo: Improving the Learning Experience of Software Tutorial Videos with Collective Interaction Data

Saelyne Yang, Jisu Yim, Aitolkyn Baigutanova, Seoyoung Kim, Minsuk Chang, Juho Kim

IUI 2022
pdf Project Website
Snapstream: Snapshot-based Interaction in Live Streaming for Visual Art

Saelyne Yang, Changyoon Lee, Hijung Valentina Shin, Juho Kim

CHI 2020
pdf Project Website Video Preview
What Makes It Hard for Users to Follow Software Tutorial Videos?

Saelyne Yang, Juho Kim

HCI Korea 2020
pdf

Experiences

  • Adobe Research
    Research Intern, STORIE Lab (May. 2024 - Aug. 2024)
    Mentors: Dingzeyu Li, Anh Truong
  • Autodesk Research
    Research Intern, HCI & Visualization Group (Jun. 2023 - Sep. 2023)
    Mentors: Jo Vermeulen, Justin Matejka, George Fitzmaurice
  • LG AI Research
    Research Intern, Advanced ML Lab (Oct. 2022 - Feb. 2023)
    Mentor: Moontae Lee
  • Adobe Research
    Research Intern, Creative Technologies Lab (Jun. 2020 - Sep. 2020)
    Mentor: Hijung Valentina Shin
  • ACRYL
    Software Engineer Intern (Jan. 2018 - Feb. 2018)

Awards & Honors

  • Outstanding Paper Award, 2nd Workshop on MMRAgI (CVPR 2026)
  • Outstanding PhD Thesis Award, School of Computing, KAIST (2026)
  • Special Recognition for Outstanding Reviews (x2) (CHI 2025)
  • AAAI-24 Student Scholar (2024)
  • Google Student Travel Grant (2023)
  • NAVER PhD Fellowship (2022)
  • Special Recognition for Outstanding Reviews (CSCW 2023)
  • Special Recognition for Outstanding Reviews (CHI 2022)
  • Inseong Fellow, KAIST (2021)
  • An Outstanding TA Award, KAIST (2021 Spring, 2022 Spring)
  • An Outstanding MS Thesis Award, KAIST (2021)
  • Kwanjeong Scholarship (2019-20)
  • Encouragement Award, SW Talent Festival (2018)
  • Excellence Award, Writing Contest, KAIST (2018)
  • Google Women Techmakers Scholarship (2018)
  • Dean's List, School of Computing, KAIST (2018 Spring)
  • LINE Scholarship (2017)
  • National Science and Technology Scholarship (2017-2018)

Education

  • Ph.D. in School of Computing, KAIST (Mar. 2021 - Feb. 2026)
    GPA: 4.16/4.3, Outstanding PhD Thesis Award
  • M.S. in School of Computing, KAIST (Mar. 2019 - Feb. 2021)
    Outstanding MS Thesis Award
  • B.S. in School of Computing, KAIST (Mar. 2015 - Feb. 2019)
    Graduated with Honors (Magna Cum Laude)