A Tree-Structured Decoder for Image-to-Markup Generation

Zhang, Jiang, Du, Jun, Yang, Yongxin, Song, Yi-Zhe, Wei, Si and Dai, Lirong (2020) A Tree-Structured Decoder for Image-to-Markup Generation In: 37 th International Conference on Machine Learning, 2020-07-13-2020-07-18, Vienna, Austria.

Preview

Text TreeDecoder.pdf - Accepted version Manuscript Download (464kB) | Preview

Abstract

Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup. However, for tree-structured representational markup, string representations can hardly cope with the structural complexity. In this work, we first show via a set of toy problems that string decoders struggle to decode tree structures, especially as structural complexity increases, we then propose a tree-structured decoder that specifically aims at generating a tree-structured markup. Our decoders works sequentially, where at each step a child node and its parent node are simultaneously generated to form a sub-tree. This sub-tree is consequently used to construct the final tree structure in a recurrent manner. Key to the success of our tree decoder is twofold, (i) it strictly respects the parent-child relationship of trees, and (ii) it explicitly outputs trees as oppose to a linear string. Evaluated on both math formula recognition and chemical formula recognition, the proposed tree decoder is shown to greatly outperform strong string decoder baselines.

Item Type:	Conference or Workshop Item (Conference Paper)
Divisions :	Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :	Name Email ORCID Zhang, Jiang Du, Jun Yang, Yongxin yongxin.yang@surrey.ac.uk Song, Yi-Zhe y.song@surrey.ac.uk Wei, Si Dai, Lirong
Date :	1 June 2020
Copyright Disclaimer :	Copyright 2020 the author(s).
Depositing User :	Diane Maxfield
Date Deposited :	07 Oct 2020 15:47
Last Modified :	07 Oct 2020 15:47
URI:	http://epubs.surrey.ac.uk/id/eprint/858707

Actions (login required)

View Item

Downloads