A Tree-Structured Decoder for Image-to-Markup Generation
Zhang, Jiang, Du, Jun, Yang, Yongxin, Song, Yi-Zhe, Wei, Si and Dai, Lirong (2020) A Tree-Structured Decoder for Image-to-Markup Generation In: 37 th International Conference on Machine Learning, 2020-07-13-2020-07-18, Vienna, Austria.
|
Text
TreeDecoder.pdf - Accepted version Manuscript Download (464kB) | Preview |
Abstract
Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup. However, for tree-structured representational markup, string representations can hardly cope with the structural complexity. In this work, we first show via a set of toy problems that string decoders struggle to decode tree structures, especially as structural complexity increases, we then propose a tree-structured decoder that specifically aims at generating a tree-structured markup. Our decoders works sequentially, where at each step a child node and its parent node are simultaneously generated to form a sub-tree. This sub-tree is consequently used to construct the final tree structure in a recurrent manner. Key to the success of our tree decoder is twofold, (i) it strictly respects the parent-child relationship of trees, and (ii) it explicitly outputs trees as oppose to a linear string. Evaluated on both math formula recognition and chemical formula recognition, the proposed tree decoder is shown to greatly outperform strong string decoder baselines.
Item Type: | Conference or Workshop Item (Conference Paper) | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Divisions : | Faculty of Engineering and Physical Sciences > Electronic Engineering | |||||||||||||||||||||
Authors : |
|
|||||||||||||||||||||
Date : | 1 June 2020 | |||||||||||||||||||||
Copyright Disclaimer : | Copyright 2020 the author(s). | |||||||||||||||||||||
Depositing User : | Diane Maxfield | |||||||||||||||||||||
Date Deposited : | 07 Oct 2020 15:47 | |||||||||||||||||||||
Last Modified : | 07 Oct 2020 15:47 | |||||||||||||||||||||
URI: | http://epubs.surrey.ac.uk/id/eprint/858707 |
Actions (login required)
![]() |
View Item |
Downloads
Downloads per month over past year