Abstract
Habitual knowledge of everyday activities (e.g. going to a restaurant) plays a key role in our interaction with the environment and in language comprehension. Such events and their participants are stored in our semantic memory (as script knowledge) in an order-sensitive way: the order of mention of script events has been shown to determine if following events are cued (e.g. cook, sit → dine). Language also uses discourse connectors, and temporal ones in particular (before / after), to signal possible reordering of the events. We then assume that temporal connectives trigger expectations about the events’ order of mention, generating predictions about what event will be mentioned next. According to the iconicity assumption, processing is easier when the order of mention matches the chronological order of the events, as no reordering is needed. Previous work has shown the effect of fast integration of discourse content and connectives, but the early integration of discourse connectives and script knowledge has not been shown before. Our hypothesis is that people make an early use not only of discourse connectives but also of broader world knowledge information about scripts as soon as context cues become available, to quickly build expectations about what will be mentioned next.
In the present visual world study participants heard short stories, each about a particular scenario, followed by test sentences such as: (a) Before / After the meal read he eagerly the menu / the bill (b) Before / After the meal ordered he eagerly an appetizer / a dessert. Along with the sentences, participants are presented with visual scenes, each containing six objects: one target object (e.g. menu), one temporal-order distractor, which holds the same temporal order relation (before or after) with the main event as the target (e.g. appetizer, referring to another before-event), one selectional-restriction distractor, matching the same main verb as the target (e.g. bill), and two fillers, which were congruent with the script, but did not match the verb nor were participants in an obvious before- or after-event. We analyze three regions: the before/after region (starting 200 ms into the main event, e.g. dem Essen), the verb region (starting 200 ms into the verb onset, e.g. las), and the object region (starting 200 ms into the object’s determiner, e.g. die Speisekarte). At the before/after region, we expect the connectors to cue the objects matching the participant’s script knowledge of what event comes before or after the main event, resulting in more gazes at the target and at the temporal order distractor compared to the other objects. Following the iconicity assumption, we expect the effect of the discourse connector to be stronger for the after condition compared to the before condition. At the verb region, we expect the verb’s selectional restrictions to interact with the temporal discourse cues, restricting the comprehender’s prediction to the target object, that is the only object matching both the temporal order cues and the verb’s selectional restrictions.
Preliminary results (N = 16) show an increased proportion of first fixations on the objects matching the temporal order (target + temporal-order distractor) at the before/after region for the after condition, but not for the before condition. First fixations on the objects matching the verb (target + selectional-restriction distractor) are increased at the verb region for both conditions. The event order cues and the verbal cues are integrated as early as the verb region, where we observe overall increased first fixations on the target. Unsurprisingly, first fixations on the target are also increased at the object. The advantage of the after-condition shows that people have a natural tendency to anticipate the next script event before it is mentioned, rather than the previous one, even when this is cued by context. Our results are compatible with expectation-based views of language processing, as they show that people quickly integrate linguistic cues (relative to the verb’s preferences) and world knowledge about habitual events and their typical order as soon as such cues become available.
Alessandra Zarcone
Professor of Language Technologies and Cognitive Assistants
Computational linguist with a background in NLP and in psycholinguistics, working on AI, NLP and human-machine interaction.