ACM SIGCHI General Interest Announcements (Mailing List)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Adam Jatowt <[log in to unmask]>
Reply To:
Adam Jatowt <[log in to unmask]>
Sat, 15 Oct 2016 23:54:16 +0900
text/plain (130 lines)
+++Apologies for cross-posting+++


NTCIR-13 Actionable Knowledge Graph (AKG) Task

The dry run of our first subtask is now ready to start!
Oct 15, 2016: Action Mining: Dry run topics + sample data release
Nov 15, 2016: Action Mining: Dry run submissions due
Dec 1, 2016: Action Mining: Dry run results release

The “dry run” at NTCIR is a great opportunity to obtain initial evaluation
results that can optimise your system for formal run submissions, so don’t
miss it.

Join us today:

=== OVERVIEW ===
Knowledge graph (aka direct displays, cards, etc.) has become an
increasingly common and important component in search engine result pages
(SERPs). The objective of this new pilot task accepted to run at NTCIR-13
is to foster research on generating knowledge graphs that are optimised for
facilitating users' actions (buying, booking, downloading, comparing,
creating, etc.). To achieve this objective, we set two subtasks: Action
Mining (AM) subtask and Actionable Knowledge Graph Generation (AKGG)

=== TASKS ===
In this pilot task, we set two subtasks to achieve and advance the
technologies related to actionable knowledge graph presentations that can
be used for search engines.

- Action Mining Subtask (AM)
Input: Entity, Entity Type, and Wikipedia URL
Output: A ranked list of actions

For a given entity type (e.g., Place) and instance entity (e.g., "poland"),
participants will be asked to find potential actions that can be taken
(e.g., "visit poland", "buy a house in poland", "find weather in poland").
Participants are allowed to use any external resources (e.g., Action
section in to return a ranked list of potential actions. Up to
100 actions should be submitted for each query (pair of entity and its
type). The format of each submitted action should contain verb and object
(called also the modifier of a verb), where the object's length is limited
to 50 characters. For example, in the above-mentioned case of "poland" as
the entity instance of the Place type, "buy a house in poland" would be an
action composed of a verb "buy" and object "a house in poland".
Participants are allowed to submit up to three actions that share the same
verb. Note that actions can sometimes lack objects (actions containing only
a verb), in which case, the object is actually considered NULL. For
example, for the entity "outlook express" of the type Product, "download"
is an example of correct action that does not require any explicit object
This subtask can be seen as an open information extraction task, and allows
us to accumulate a comprehensive set of actions that are related to a given
entity type and entity instance. The returned actions will be assessed by
crowdsourcing to be scored from 1 to 5. Actions will be evaluated not only
based on their relevance but also based on diversity in order to prevent
submissions of many similar actions (e.g., actions where verbs are synonyms
or the objects have very similar meaning).
For sample data see:

- Actionable Knowledge Graph Generation Subtask (AKGG)
Input: Query, Entity Type, Entity, and Action
Output: A ranked list of attributes of the type

For a given search query, entity included in that query, the type of the
entity, and action (e.g., "kyoto budget travel", "kyoto", location, "visit
a temple"), participants will be asked to rank entity properties based on
their relevance to the query. The query (input) can be ambiguous as in
realistic search queries, and participants need to return the ranked list
of relevant entity properties to create an actionable knowledge graph.
Actions in the test queries will be taken from the outcomes of the Action
Mining (AM) Subtask. Properties to be returned will be those defined as
attributes of the entity type in vocabulary.

Please note:

Task participants must submit a paper to the NTCIR-13 Conference
(unrefereed) and at least one member of each participating group must
attend the conference (December 5-8, 2017, NII, Tokyo, Japan) to present
their work.

Please visit task website for more information:

Oct 15, 2016: AM: Dry run topics + sample data release
Nov 15, 2016: AM: Dry run submissions due
Dec 1, 2016: AM: Dry run results release
Dec 15, 2016: Task registration due
Jan 1, 2017: AM: Formal run topics release
Feb 1, 2017: AM: Formal run submissions due
Feb 15, 2017: AM: Formal run results release
Mar 15, 2017: AKGG: Dry run topics + training data release
Apr 15, 2017: AKGG: Dry run submissions due
May 15, 2017: AKGG: Dry run results release
Jun 15, 2017: AKGG: Formal run topics release
Jul 15, 2017: AKGG: Formal run submissions due
Sep 1, 2017: AKGG: Formal run results release & Partial Overview paper
Oct 1, 2017: Participants’ papers due
Nov 1, 2017: Camera-ready papers due
Dec 5-8, 2017: NTCIR-13 conference @ NII, Tokyo, Japan

Roi Blanco (University of A Coruña, Spain)
Hideo Joho (University of Tsukuba, Japan)
Adam Jatowt (Kyoto University, Japan)
Haitao Yu (University of Tsukuba, Japan)
Shuhei Yamamoto (University of Tsukuba, Japan)

=== CONTACT ===
ntcir-akg at

    For news of CHI books, courses & software, join CHI-RESOURCES
     mailto: [log in to unmask]

    To unsubscribe from CHI-ANNOUNCEMENTS send an email to
     mailto:[log in to unmask]

    For further details of CHI lists see