Pixelor: A Competitive Sketching AI Agent. So you think you can beat me?

Bhunia, Ayan Kumar, Das, Ayan, Muhammad, Umar Riaz, Yang, Yongxin, Hospedales, Timothy M., Xiang, Tao, Gryaditskaya, Yulia and Song, Yi-Zhe (2020) Pixelor: A Competitive Sketching AI Agent. So you think you can beat me? ACM Transactions on Graphics, 39 (6).

Preview

Text Pixelor - AAM.pdf - Accepted version Manuscript Download (11MB) | Preview

Abstract

We present the first competitive drawing agent Pixelor that exhibits human-level performance at a Pictionary-like sketching game, where the participant whose sketch is recognized first is a winner. Our AI agent can autonomously sketch a given visual concept, and achieve a recognizable rendition as quickly or faster than a human competitor. The key to victory for the agent’s goal is to learn the optimal stroke sequencing strategies that generate the most recognizable and distinguishable strokes first. Training Pixelor is done in two steps. First, we infer the stroke order that maximizes early recognizability of human training sketches. Second, this order is used to supervise the training of a sequence-to-sequence stroke generator. Our key technical contributions are a tractable search of the exponential space of orderings using neural sorting; and an improved Seq2Seq Wasserstein (S2S-WAE) generator that uses an optimal-transport loss to accommodate the multi-modal nature of the optimal stroke distribution. Our analysis shows that Pixelor is better than the human players of the Quick, Draw! game, under both AI and human judging of early recognition. To analyze the impact of human competitors’ strategies, we conducted a further human study with participants being given unlimited thinking time and training in early recognizability by feedback from an AI judge. The study shows that humans do gradually improve their strategies with training, but overall Pixelor still matches human performance. The code and the dataset are available at http://sketchx.ai/pixelor.

Item Type:	Article
Divisions :	Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :	Name Email ORCID Bhunia, Ayan Kumar a.bhunia@surrey.ac.uk Das, Ayan a.das@surrey.ac.uk Muhammad, Umar Riaz u.muhammad@surrey.ac.uk Yang, Yongxin yongxin.yang@surrey.ac.uk Hospedales, Timothy M. Xiang, Tao t.xiang@surrey.ac.uk Gryaditskaya, Yulia y.gryaditskaya@surrey.ac.uk Song, Yi-Zhe y.song@surrey.ac.uk
Date :	December 2020
DOI :	10.1145/3414685.3417840.
Copyright Disclaimer :	© 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics, https://doi.org/10.1145/3414685.3417840.
Uncontrolled Keywords :	Sketch-generation; Neural search; Early recognition; AI games; Recurrent neural network
Related URLs :	http://tog.acm.org/
Depositing User :	Clive Harris
Date Deposited :	07 Oct 2020 08:57
Last Modified :	07 Oct 2020 08:57
URI:	http://epubs.surrey.ac.uk/id/eprint/858688

Actions (login required)

View Item

Downloads