CLUNL

TKB – Transmedia Knowledge Base for Contemporary Dance

Identification

• Project identification: TKB – Transmedia Knowledge Base for Contemporary Dance                                                  [ PTDC/EAT-AVP/098220/2008 ]
• Group: Lexicology, Lexicography and Terminology
  
• Principal Investigator: Carla Fernandes
  
• Duration: Feb. 2010 – Jan. 2013
  
• Funding entity: Portuguese Foundation for Science and Technology  (FCT, I.P.)
• Website: http://tkb.fcsh.unl.pt/

Description

In the interstices between linguistics and contemporary dance studies and continuing the research line of the VILA_CCC_Mont (project funded by a Post-doc grant awarded to the project coordinator), TKB is a more extensive and transdisciplinay project aiming at the design and construction of an open-ended multimodal knowledge base to document, annotate and support the creation of contemporary dance pieces.

The TKB project is planned to run from 2009 to 2012 and seeks to provide a research space for rigorous, critical exploration of the relationship between linguistics, dance studies, new digital media and thought/consciousness.

Its main targets are: to extend the scope and application of the “documentation” concept to contemporary dance in different ways; to develop a strong link between the recent dance-research community and the well-established communities in cognitive linguistics (since Lakoff & Johnson 1980) and computer science, by taking a closer look at the cognitive process of “choreographic thinking” (Stevens & McKechnie 2005) and therefore contribute to the domains of multimodal corpora (Kipp 2008), terminological ontologies, cognition and verbal-nonverbal relations.

TKB implies the articulation of 3 complementary components, each of them involving specific aims and tasks, as specified below.

1. Linguistic annotation

Video annotation of multiple rehearsal videos
Creation of specialized glossary to define the terms used
Prototype of online archive for Portuguese contemporary dance

The aim of this component is to linguistically annotate original videos from dance performances shot on site during their creation and rehearsal phases, in a first stage those corresponding to the piece SetUp by Rui Horta, being contrasted with two other pieces by the same creator, with which it is considered to form a trilogy: Pixel, SetUp and Scope. This will be tested as a first case study, other choreographers being planned for analysis as well, both for contrasting and evaluation purposes.

A terminological glossary of choreographic elements used by Rui Horta and his dancers is intended to be a rather flexible tool, in the sense that its underlying structure will be adapted to accommodate other choreographer-related terms as well, as is the intention of our partner Bertha Bermudez while working in a similar project with the dance company Emio Greco | PC.

With the prototype for a Digital Archive of Portuguese contemporary dance creations, which should be seen as a work-in-progress, our aim is to gradually compile in an interactive Transmedia Knowledge Base the newest creations of international choreographers showing interest in this initiative, as well as, in the longer run, to cover the existing resources documenting the choreographic productions of the last three decades in Portugal. The archive should also include the existing press and video documentation, as well as a spectator’s gallery for feed-back and public intervention.

The linguistic annotation component will be developed both at CLUNL and CLUP with the collaboration of the Amsterdam School of Arts (AHK), under the supervision of Carla Fernandes.

2. Software development for annotation and motion analysis

Video annotator
Information & Knowledge Management
Motion analysis research

This component includes the creation of a custom-made video annotator, an information & knowledge management system as the interface to connect its three different modules and a motion analysis research module: towards a multi-person semi-automatic system to analyze the human body motion in video dance sequences (Guo 2006). This corresponds to two different applications: one for annotation and another one, web-based, for content access and browsing.

This software development component will be developed at FCT/UNL, Interactive Multimedia Group, under the supervision of Nuno Correia.

3. Creation-oriented tool

Tool for choreographers’ creation process and individual archives
Design of the TKB interfaces

The novelty here is to design a creation-oriented tool deriving from the results of the annotated video corpora with the aim of feeding back to the choreographic creative process. The interfaces designed in this component will be implemented in component 2. The Web interface has different modes, based on the case studies, for different types of users, e.g., choreographers or archive users. It should be an auxiliary tool for the creator’s task in real time compositions and for their archiving intentions (Jürgens 2008).

This will be developed by new media designer and artist Chris Ziegler, with the collaboration of Stephan Jurgens and Rui Horta.

We will have a contemporary dance-expert consultant (Maria José Fazenda) and a facilitator consultant for the project´s knowledge circulation (Scott DeLahunta).

TKB is to be launched in the web, being relevant for dance companies, higher education dance schools, contemporary art museums and for the scientific communities of cognitive linguistics, multimodal corpora and new media technologies.

Team

Researchers:
Carla Fernandes (principal investigator)
Nuno Correia (Faculty of Sciences and Technology, Universidade NOVA de Lisboa – NOVA FCT)
Isabel Galhano (Linguistics Centre, Universidade do Porto – CLUP)
Rute Costa
Stephan Jürgens
Bertha Bermudez (AHK – Academy of Theatre and Dance, Amsterdam University of Arts)
David Santos (research grant)
Evi Dimakopoulou (CLUP, research grant)
João Gaspar (NOVA FCT, research grant)
João Santos (NOVA FCT, research grant)
Diogo Cabral (NOVA FCT, research grant)
João Silva (NOVA FCT, research grant)
Urândia Aragão (NOVA FCT, research grant)

Consultants:
Maria José Fazenda (Escola Superior de Dança de Lisboa)
Scott DeLahunta (Motion Bank Institute, Frankfurt)

Participating entities

Interactive Multimedia Group, Universidade NOVA de Lisboa
Centro Coreográfico “O Espaço do Tempo”
Linguistics Centre, Universidade do Porto
AHK – Academy of Theatre and Dance, Amsterdam University of Arts

Associate partners

Centro Cultural de Belém
Atelier Re.Al

Publications

Articles

Fernandes, Carla and Bermudez, Bertha (2010). Inventing the interactive glossary: an approach to documenting contemporary dance. RTRSRCH Journal, (Special issue: Notation), pp.29-31.

Fernandes, Carla and Costa, Rute (2010). Looking for the linguistic knowledge behind the curtains of contemporary dance: the case of Rui Horta’s creative process. International Journal of Arts and Technology, 3(2-3), pp.235–250. ISSN 1754-8853.

Fernandes, Carla and Jürgens, Stepahn (2013). Video annotation in the TKB project: linguistics meets choreography meets technology. International Journal of Performance Arts & Digital Media, [online] 9(1), pp.115-134. Available at: http://www.tandfonline.com/doi/abs/10.1386/padm.9.1.115_1 [Accessed 28 Sep. 2017]. ISSN 1479-4713. DOI 10.1386/padm.9.1.115_1

Jesus, Rui; Abrantes, Arnaldo and Correia, Nuno (2011). Methods for automatic and assisted image annotation. Multimedia Tools and Applications, [online] 55(1), pp.7-26. Available at: https://link.springer.com/article/10.1007/s11042-010-0586-z [Accessed 20 Nov. 2017]. e-ISSN 1573-7721; ISSN 1380-7501.

Chapters in books/proceedings

Fernandes, Carla (2013). The TKB Project: Creative Technologies for Performance Composition, Analysis and Documentation. In: P. Nesi and R. Santucci, eds., Information Technologies for Performing Arts, Media Access, and Entertainment. Second International Conference, ECLAP 2013. Revised Selected Papers. London: Springer, pp.205-217. e-ISBN 978-3-642-40050-6; ISBN 978-3-642-40049-0.

Cabral, Diogo; Carvalho, Urândia; Silva, João; Valente, João; Fernandes, Carla and Correia, Nuno (2011). Multimodal video annotation for contemporary dance creation. In: Extended Abstracts on Human Factors in Computing Systems (CHI EA ’11). New York: ACM, pp.2293-2298. Available at: https://dl.acm.org/citation.cfm?id=1979930 [Accessed 20 Nov. 2017]. ISBN 978-1-4503-0268-5. DOI 10.1145/1979742.1979930

Cabral, Diogo; Valente, João; Aragão, Urândia; Fernandes, Carla and Correia, Nuno (2012). Evaluation of a multimodal video annotator for contemporary dance. In: G. Tortora, S. Levialdi and M. Tucci, eds., Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI’12). [PDF] New York: ACM, pp.572-579. Available at: https://dl.acm.org/citation.cfm?doid=2254556.2254663 [Accessed 11 Oct. 2017]. ISBN 978-1-4503-1287-5. DOI 10.1145/2254556.2254663

Cabral, Diogo; Valente, João; Silva, João; Aragão, Urândia; Fernandes, Carla and Correia, Nuno (2011). A Creation-Tool for Contemporary Dance using Multimodal Video Annotation. In: Proceedings of the ACM internacional conference on Multimedia (MM’11). [PDF] New York: ACM, pp.905-908. Available at: https://dl.acm.org/citation.cfm?id=2071899 [Accessed 4 Oct. 2017]. ISBN 978-1-4503-0616-4. DOI 10.1145/2072298.2071899

Fernandes, Carla and Jürgens, Stephan (2010). Transdisciplinary research bridging cognitive linguistics and digital performance: from multimodal corpora to choreographic knowledge-bases. In: F. Schroeder, ed., Performing Technology: User Content and the New Digital Media. Newcastle upon Tyne: Cambridge Scholars Publishing, pp.19-34. ISBN 978-1-4438-1445-4.

Realinho et al. (2011). Building mobile context-aware applications for leisure and entertainment. In: Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE ’11). New York: ACM, [n.d.].

Silva, João; Cabral, Diogo; Fernandes, Carla and Correia, Nuno (2012). Real-Time Annotation of Video Objects on Tablet Computers. In: Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM’12). [PDF] New York: ACM, 19:1-19:9. Available at: https://dl.acm.org/citation.cfm?doid=2406367.2406391 [Accessed 4 Oct. 2017]. ISBN 978-1-4503-1815-0. DOI 10.1145/2406367.2406391

Theses

Gaspar, João (2011). Sistema Multimodal para captura e anotação de vídeo. Master. Supervised by Nuno Correia. Universidade NOVA de Lisboa. Available at: http://hdl.handle.net/10362/6633 [Accessed 20 Nov. 2017].

Jürgens, Stephan (2011). A methodology for Bi-directional transfer between Contemporary Dance and New Media Technologies. PhD. Supervised by Daniel Tércio Ramos Guimarães. Universidade de Lisboa. Available at: http://hdl.handle.net/10400.5/3775 [Accessed 20 Nov. 2017].

Santos, João (2012). Relational navigation and archiving of multimedia information for contemporary dance. Master. Supervised by Nuno Correia. Universidade NOVA de Lisboa. Available at: http://hdl.handle.net/10362/8449 [Accessed 20 Nov. 2017].

Silva, João (2012). People and object tracking for video annotation. Master. Supervised by Nuno Correia. Universidade NOVA de Lisboa. Available at: http://run.unl.pt/handle/10362/8953 [Accessed 20 Nov. 2017].

Santos, David (2011). Dramaturgia Multimodal: Anotação digital de corpora multimodais. Master. Supervised by Carla Fernandes. Universidade do Porto. Available at: https://repositorio-aberto.up.pt/bitstream/10216/61483/1/000148974.pdf [Accessed 20 Nov. 2017].

Dimakopoulou, Paraskevi (2011). Towards the creation of an annotation system and a digital archive platform for contemporary dance. Master. Supervised by Nuno Correia; CoSupervised by Isabel Galhano. Universidade do Porto. Available at: https://repositorio-aberto.up.pt/bitstream/10216/61522/1/000148969.pdf [Accessed 20 Nov. 2017].