Korea Academic Institute of Science and Technology

In this paper, we introduce Attentive Guidance (AG), a new mechanism todirect a sequence to sequence model equipped with attention to find morecompositional solutions that generalise even in cases where the training andtesting distribution strongly diverge. We test AG on two tasks, devisedprecisely to asses the composi- tional capabilities of neural models and showhow vanilla sequence to sequence models with attention overfit the trainingdistribution, while the guided versions come up with compositional solutionsthat, in some cases, fit the training and testing distributions equally well.AG is a simple and intuitive method to provide a learning bias to a sequence tosequence model without the need of including extra components, that we believeallows to inject a component in the training process which is also present inhuman learning: guidance.