Wednesday, February 29, 2012

Available Open Source Dialogue Systems Frameworks

Once in a while someone asks me if there is any dialogue system framework s/he can use out of the box.
Recently I put this one together:
  • Olympus
    CMU LTI, Dan Bohus (RavenClaw)
    Olympus is an architecture for spoken dialog system created at Carnegie Mellon University (CMU). It is mainly designed to help researchers in conversational agents implement and test their ideas on full systems, without having to build them from scratch.
    The Olympus architecture (Bohus et al. 2007) incorporates a number of modules developed at CMU in past and current spoken dialogue systems research projects.

  • Galatea Project
    University of Japan
    Galatea is a project for providing an open-source, license-free software toolkit for building anthropomorphic spoken dialogue agents. In other words, using this toolkit, you can build your own unique life-like visual agent that can communicate with you via spoken language. (note: Anthropomorphic agent is being called in various ways: digital human, virtual human, avatar, animated agent, life-like agent, human-like agent, talking head, animated human image, ... etc. with slightly different emphasis to its various aspects.)

  • Regulus
    NASA
    Regulus is a Prolog-based toolkit for building spoken dialogue systems.

  • The SEMAINE project
    The aim of the SEMAINE project is to build a Sensitive Artificial Listener – a multimodal dialogue system with the social interaction skills needed for a sustained conversation with a human user. The system will emphasise “soft” communication skills, i.e. non-verbal, social and emotional perception, interaction and behaviour capabilities. The Sensitive Artificial Listener paradigm involves only very limited verbal capabilities, but has been shown to be suited for prolonged human-machine interaction. In this paradigm, we will build a real-time, robust interactive system perceiving a human user's facial expression, gaze, and voice, and engaging with the user through an Embodied Conversational Agent's body, face and voice. The agent will exhibit audiovisual listener feedback in real time while the user is speaking, and will take the user's feedback into account while the agent is speaking. The agent will pursue different dialogue strategies depending on the user's state; it will learn to interpret the user's non-verbal behaviour and adapt its own behaviour accordingly.

  • Jaspis
    University of Tampere, Finland, Markku Turunen
    Jaspis is a framework for adaptive speech applications. Jaspis is designed to support distributed, highly context-sensitive applications that adapt to the user and the environment. It is designed multilingual applications in mind. The architecture is based on the shared information management and on the agent-evaluator-manager -paradigm.

  • TrindiKit
    Göteborg University Dialogue Systems Lab, Sweden, Staffan Larsson
    TRINDI (Task Oriented Instructional Dialogue) KIT is a toolkit for building and experimenting with dialogue move engines andinformation states, that has been developed in theTRINDI and SIRIDUS projects, and is currently being developed further in the TALK project.

  • Midiki - MITRE Dialogue Toolkit,
    Midiki is a portable toolkit for building dialogue managers in Java. It implements the information-state model of dialogue as pioneered in Trindikit: a rule-based, theory-neutral, platform agnostic model.

  • WAMI: Web-Accessible Multimodal Applications
    CSAIL, MIT, USA, Alex Gruenstein, Ian McGraw
    Wami is an open source Javascript API for speech recognition. This project contains both the server and client side code necessary to host a web-based API for speech recognition. This project does not contain a speech recognizer. This project provides all the plumbing you'll need to give a speech recognizer a web-interface. There's even an iPhone app associated with it (search "wami" in the app store).

  • CSLU Toolkit
    Center for Spoken Language Understanding (CSLU) OGI Campus, Oregon Health & Science University (OGI/OHSU), John-Paul Hosom
    A Platform for Research and Development of Spoken-Language Systems
    The CSLU Toolkit was created to provide the basic framework and tools for people to build, investigate and use interactive language systems. These systems incorporate leading-edge speech recognition, natural language understanding, speech synthesis and facial animation technologies. The toolkit provides a comprehensive, powerful and flexible environment for building interactive language systems that use these technologies, and for conducting research to improve them.
    Because of the Toolkit's easy to use graphical authoring tools, it has spread from the halls of higher education into homes, primary and secondary schools. People who are currently using the Toolkit range from fifth-graders who create school projects to university professors who are experimenting with core language technologies and spoken dialogue systems.

  • Ariadne
    Matthias Denecke
    Ariadne is an open source spoken dialogue system developed as a thesis research to prove a generic task and language independent dialogue system that generates interactions with users based on a description of an application.
    The system architecture is a layered microkernel architecture which means than that you can replace pretty much every component other than the central dialogue processing component. This includes parsers, natural language generation, the mechanisms of the underlying logics and so on. The price to pay is that if you want to do that you will have to stick to the APIs provided.
    It also means that you can develop entire dialogue systems without the need for complicated middleware, socket connections and servers. This should make it easier to port this system to embedded applications.

  • DIPPER
    Dialogue Prototyping Equipment & Resources,
    DIPPER is a collaborative initiative of the Language Technology Group at The University of Edinburgh and CSLI Stanford to create a shared software database to comfort in prototyping dialogue systems, using OAAas communication protocol. DIPPER is not a dialogue system itself, but DIPPER supports building (spoken) dialogue systems, by offering interfaces to speech recognisers (Nuance), speech synthesisers (Festival), dialogue managers, natural language understanding, and automated reasoning (SPASS, Bliksem, Mace, PARADOX).

  • PED: A Planner for Efficient Dialogues
    University College Dublin, Ireland, Bryan McEleney
    PED is a dialogue management system that uses a probabilistic nested belief model to choose dialogue strategies from a Bayesian game tree. The system designer is only required to supply a context-free dialogue grammar with preconditions. Using this grammar, PED constructs game trees (like the one below) to represent the outcomes of the dialogue. PED uses a belief revision function that makes inferences from the acts observed in the dialogue to update the belief model. PED is more efficient than other systems because it uses a probabilistic belief model. However, the belief revision function described here is approximate and I have not yet seen its solution.

  • Open Allure Dialogue System
    Open Allure is a project aimed at developing new ways to share what we know with one another by permitting the collaborative creation and experience of interactive dialogs. These verbal exchanges give your interaction with the computer a very different quality and permit immediate feedback to help reinforce or reorganize your thinking. Because voice recognition is still imperfect, the interface could also support making choices by gesture: your webcam watches you as you raise your hand.
    So with the same basic hardware (headset microphone and webcam) you might use with Skype, Open Allure offers you a conversation with your computer: it talks, it listens, it watches, it responds.
    As of 2011, Open Allure is multilingual and mobile. Portuguese (pt) and Italian (it) versions are available for download. Please request other languages if you have interest. An Android version and Java source code is also available.

  • OWLSpeak, an ontology-based spoken dialogue management system
    Uml University, Germany
    The system is written in Java and uses OWL Spoken Dialogue Ontologies to generate VoiceXML dialogue documents. OwlSpeak was implemented within the framework of the EU-funded project ATRACO - Adaptive and TRusted Ambient eCOlogies.
Open source speech recognizers include Sphinx, PocketSphinx, and Flite from CMU and OpenEars.

References:

That's it! Hope this can be useful...

Tuesday, August 23, 2011

World Youth Days

Tor Vergata, Rome, Italy, 2000



Quatro Vientos, Madrid, Spain, 2011









History of WYD:

Thursday, August 4, 2011

MIT CSAIL Cooking-Baking Robot



MIT CSAIL students programmed the PR2 robot (built by Willow Garage) to perform the wide variety of tasks required to bake cookies. For more info...

Thursday, October 7, 2010

Married Life as seen on Disney Pixar UP!

Carl & Ellie by Michael Giacchino










My favorite movie!

Wednesday, March 17, 2010

Thursday, March 11, 2010

Wednesday, February 24, 2010

DIY: SHiFT'2010

A short video on SHiFT a happening, an international conference and unique event, gathering leaders and participants from all walks of life, to discuss and share Social and Human Ideas For Technology.
SHIFT 2010, 16-17th April
Teatro Aberto, Lisbon, Portugal


SHiFT 2010: DIY from Elmine Wijnia on Vimeo.


Last Lecture: Really Achieving your Childhood Dreams, by Randy Pausch

Tuesday, February 2, 2010

Situated interaction with a virtual receptionist


Related publications:
  • Bohus, D., Horvitz, E. (2009) Learning to Predict Engagement with a Spoken Dialog System in Open-World Settings, in Proceedings of SIGdial'09, London, UK (pdf from ACL anthology digital archive).

  • Bohus, D., Horvitz, E. (2009) - Models for Multiparty Engagement in Open-World Dialog, in Proceedings of SIGdial'09, London, UK (pdf from ACL anthology digital archive).
(More information here...)

AR Drone: Augmented Reality Robots Take Flight

The AR Drone is one of the first consumer products to combine robotics with augmented reality. The quad-rotor vehicle carries two cameras and can stream video to an iPhone via Wi-Fi. On the phone, the live video is overlaid with flight controls and additional graphics for a variety of games.



Friday, November 13, 2009

Long time, no see...

It's been a long time since I've been here last time...
In the meanwhile, I've been to Dusseldorf to visit Pedro and Patricia, then to Brighton for Interspeech'2009, then to London for SIGDial 2009 and YRRSDS 2009.
Back to Lisbon I've been lecturing a lot of classes (16.5 hours a week) and office hours (9 a week).
I'm also coding my first baseline of my cooking coach spoken dialogue system in the context of my PhD.
Olympus, Audimus and OntoChef have been around along with Windows XP, Windows 7 and Snow Leopard, living together through Parallels.
It's being a hard a busy time, but some results are coming to light.
More news soon...

Information, by MAYAnMAYA

Nice video about what information is:

Information from MAYAnMAYA on Vimeo.

Thursday, November 5, 2009

Gelados Santini

Santini's ice creams have been one of my favorite ever since I was introduced to this small but excellent Italian ice creams factory.
It is in Cascais, Portugal, so you can grab an ice-cream and just eat it while walking trough the villa or while you watch the sea...
Happy 60 anniversary!

Tuesday, August 25, 2009

Friday, July 31, 2009

Monday, July 20, 2009

Tuesday, July 14, 2009

MIT Media Lab, LabCast #38: Robots to the Rescue

MIT Media Lab, LabCast #38: Robots to the Rescue

The Personal Robots group presents a scenario demonstrating mixed-initiative human-robot teaming for disaster response.

Monday, June 22, 2009

[PT] [Querido mudei a Casa] da Ana - Diana

Duas amigas do Liceu. Uma surpreendeu a outra...

Friday, June 19, 2009

Japanese cooking robot

What's cooking at Tokyo's International Food Machinery and Technology Expo?



More here (from Reuters)

It's good to see that some progresses were made since Johnny 5...

Thursday, June 18, 2009

Tuesday, June 16, 2009

Monday, June 15, 2009

Official MDS (Mobile Dexterous Social) Robot


Designed and built in a collaborative work by MIT Media Lab's Personal Robots Group, University of Massachusetts at Amherst's Laboratory for Perceptual Robotics, Xitome Design, and Meka Robotics.