MIWA - Project Page

Abstract

Web automation is frequently used by data scientists, domain experts, and programmers to complete timeconsuming data collection tasks. However, developing web automa- scientists, domain experts, and programmers to complete timeconsuming data collection tasks. However, developing web automa- tion scripts requires familiarity with a programming language and HTML, which remains a key learning barrier for non-expert users. This work is licensed under a Creative Commons Attribution International We provide MIWA, a mixed-initiative web automation system that enables users to create web automation scripts by demonstrating what content they want from the targeted websites. Compared to ex- existing web automation tools, MIWA helps users better understand a generated script and build trust in it by (1) providing a step-by-step explanation of the script’s behavior with visual correspondence to the target website, (2) supporting greater autonomy and control over web automation via step-through debugging and fne-grained demonstration refnement, and (3) automatically detecting potential corner cases that are handled improperly by the generated script. We conducted a within-subjects user study with 24 participants and compared MIWA with Rousillon, a state-of-the-art web automation tool. Results showed that, compared to Rousillon, MIWA reduced

The user interface of MIWA. In addition to scraping data (b and g), users can fll out a text feld (e.g., a search bar) on a target website by dragging values from a csv fle uploaded to MIWA (a). MIWA provides a step-by-step explanation of the synthesized script with visual correspondence to the UI elements in the target website (d and h). Users can validate the script behavior and inspect intermediate automation results through a step-by-step debugging feature (e). If users spot any error, they can refne the demonstration trace by editing previous actions or demonstrating new actions (c). In the end, users can scrape the data all at once by clicking the AUTO-SCRAPE button (f).

Citation

@inproceedings{chen2023miwa,
  title={Miwa: Mixed-initiative web automation for better user control and confidence},
  author={Chen, Weihao and Liu, Xiaoyu and Zhang, Jiacheng and Lam, Ian Iong and Huang, Zhicheng and Dong, Rui and Wang, Xinyu and Zhang, Tianyi},
  booktitle={Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology},
  pages={1--15},
  year={2023}
}

MIWA: Mixed-Initiative Web Automation for Beter User Control and Confidence

Abstract

Citation