Handwriting recognition

Finally, the paper, I wrote about in my last post was submitted today. The date was postponed twice. That’s why I did not have much time the last month.

It was a lot of hard work. Especially the evaluation of the webservice, I implemented. I performed a lot of tests and repeated them very often, because of some bugs in mine and in other’s source code.

Now, since this chapter is closed, I can fully concentrate on my diploma thesis. I startet writing nearly two weeks ago. The output is  rather small :( and I’m still not in a “flow”, but hopefully it will be better in some days.

The webservice is almost complete, just some little things that need to be changed after the evaluation. The only thing that is left for implementing is the ebook reader iLiad. I already startet to code some stuff, but the hardest work, the GUI, is still to come.

Today I prepared data for the evaluation of my webservice. I took some texts from http://www.bbc.co.uk/music and extracted rdf data of this service as well as from dbpedia (http://dbpedia.org/).

In the next two weeks I want to let 15 persons write five texts each and addidionally some notes. To evaluate the results will be very interesting to me. I’m really excited, how the results will look like and how I can (hopefully) improve them by changing the algorithm inside my webservice.

When I had a call to my supervisor the last time, we talked about  a publication again. He told me that there is a conference for handwriting recognition (see http://www.isical.ac.in/~icfhr2010/index.html). If I had first results of my evaluation I could write a paper and submit it there. That would be really great.  Especially because this is a specialized conference. Deadline for submitting is February, 15th. So I have to hurry up, to get this done. Of course that means a lot more work to be done, but it could help me in future.

I never wrote about what will happen with the text, the user chooses to be recognized right.

There are three possible actions the user can perform:

1. save the information
2. discard the information
3. use the text as a new container name (see below)

The webservice saves some kind of container. This is called a “research box” (or short rebo :), because it will contain any information that Robert (the journalist) wants to save for one research topic.

If he writes something onto the iLiad and sends this text to the webservice, he will get back information about the text. Then he chooses the right text and the semantic information he wants to save for later use. These are sent back to the webservice. Now this one can save everything in the research box. Therefore it uses a rdf format called TriG (see: http://www4.wiwiss.fu-berlin.de/bizer/TriG/). TriG is a compact format that contains so called named graphs (http://www.w3.org/2004/03/trix/). Every note Robert chooses for inserting into the research box will be a new graph.

As mentioned above the research box has a topic it belongs to. So it needs a name that differs for every topic. This name is the “reboname”. The user can choose a written text to be the name of the rebo. If it already exists, new data will be appended to it, otherwise a new empty container will be created.

In this week I still worked on the webservice and its interface. It becomes more and more stable :) Beside that I had to write an application that will handle the communication between the iLiad and my webservice. I simply called this “Bridge”. Why is this necessary? There are two reasons:

1. The webservice should be as general as possible. That means that is mustn’t only work with the iLiad. So there’s the need for a “translator” between both.
2. The webservice is called via XML-RPC which is a synchronous protocol. That would mean that the iLiad had to wait for the webservice and the user could not proceed in working with it. Therefore I introduced the bridge. iLiad and Bridge “talk” to each other by XMPP, which is a asynchronous chat protocol. Thus the user can send data to the webservice and is able to proceed working immediately.

It was really fun for me to implement this bridge, because I could test it by using a chat application and writing messages to it. Yes, I am talking with my own application ;)

Another advantage of this proceeding is that any component can be tested alone. I don’t need the iLiad for testing the webservice or the bridge and I don’t need the webservice for testing the iLiad.

The last few days I was modifying the interface of my webservice. That became necessary, because iDocument returns a lot more information than I expected. But how to find out, what is important to the user and what isn’t? The goal is to do as much automatic as possible. I don’t want the user to interact more than necessary.

Currently the webserver sends every word that was recognized by the hwr to iDocument. After that it does a re-ranking of the alternatives. Therefore it takes the confidence values the hwr calculated for each word  and the belief values of iDocument. In the next post I will show an example. But first I want to write about another problem: There is much information that need to be visualized on the iLiad. For now I don’t have a clue, how to do this, because there is only a greyscale display and the device is very slow.