University of Surrey

Test tubes in the lab Research in the ATI Dance Research

A Tree-Structured Decoder for Image-to-Markup Generation

Zhang, Jiang, Du, Jun, Yang, Yongxin, Song, Yi-Zhe, Wei, Si and Dai, Lirong (2020) A Tree-Structured Decoder for Image-to-Markup Generation In: 37 th International Conference on Machine Learning, 2020-07-13-2020-07-18, Vienna, Austria.

TreeDecoder.pdf - Accepted version Manuscript

Download (464kB) | Preview


Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup. However, for tree-structured representational markup, string representations can hardly cope with the structural complexity. In this work, we first show via a set of toy problems that string decoders struggle to decode tree structures, especially as structural complexity increases, we then propose a tree-structured decoder that specifically aims at generating a tree-structured markup. Our decoders works sequentially, where at each step a child node and its parent node are simultaneously generated to form a sub-tree. This sub-tree is consequently used to construct the final tree structure in a recurrent manner. Key to the success of our tree decoder is twofold, (i) it strictly respects the parent-child relationship of trees, and (ii) it explicitly outputs trees as oppose to a linear string. Evaluated on both math formula recognition and chemical formula recognition, the proposed tree decoder is shown to greatly outperform strong string decoder baselines.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Zhang, Jiang
Du, Jun
Wei, Si
Dai, Lirong
Date : 1 June 2020
Copyright Disclaimer : Copyright 2020 the author(s).
Depositing User : Diane Maxfield
Date Deposited : 07 Oct 2020 15:47
Last Modified : 07 Oct 2020 15:47

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800