The talking (and listening) Gecko

Here's the basic layout of the system.

The layer will accept inputs in two forms-

1. Regular text
2. Speech

The speech input will first have to be converted to text
using a speech recognition system called Sphinx. Since this
conversion is usually error prone, the text will be enhanced
using knowledge of the system.

After this, it can be handled in a similar way to regular text input.

At the first phase, a parser will generate a tree and tags for
a given user command. For this, a statistical parser written in
Java at the Stanford University Natural Language Group will be used, it can be checked here.

After this, the analyzer will try to determine the kind of action the user wants to perform and then the application specific interpreter will try to find the arguments in the natural language text, for example if a user wants to play some music, the title, artist, genre etc will probably be mentioned in the text, that will have to be mined.

Several times, the system will not be completely sure of the result generated, hence user recommendation will be taken to improve the accuracy.