Step 1. Generate a thesaurus by using a linked data source like DBpedia

As recently reported SWC has developed a tool called SKOSsy which can be used to extract seed thesauri from DBpedia. In our example I will generate a knowledge model describing the domain of “digital photography“. This step took around 15 minutes.

Step 2. Load the thesaurus into PoolParty and improve it to your needs

After the seed thesaurus has been loaded into PoolParty Thesaurus Manager you have many possibilities to enhance the knowledge model further: Add more categories, synonyms, relations etc. In this example I use the seed-thesaurus without any further improvements. This step took approximately 2 minutes.

Step 3. Generate an automatic text extractor on top of your thesaurus

This step took a couple of seconds and ended up in having generated a fast and reliable text mining application on top of PoolParty Extractor, ready to be used to enrich your documents with data from the LOD cloud.

You can try it out here: PPX Live-Demo

To try the extractor on your own, please take a look at the image above which shows a proper configuration, you have to insert the following UUID in the form: d35d4ddb-adc3-4ea5-b027-deacac03e391

Since our example is all about ‘digital photography’, we recommend to use text samples (or some fragments) like these ones to test the quality of PPX based text analytics:

• Digital Camera Image Noise (Results as HTML, RDF/XML)
• Nikon D3S In-depth Review (Results as HTML, RDF/XML)
• Introduction to Shutter Speed in Digital Photography (Results as HTML, RDF/XML)
• Digital Camera Sensors (Results as HTML, RDF/XML)

Let us know what you think about this straight-forward approach and your opinion about the quality of the results. We believe that thesaurus based text mining is in many cases an alternative to some other approaches, especially if you want to to enrich your content with information from the upcoming web of data.

Of course we would be happy to generate other demos in the areas of your interest! Just get in contact with us by using our contact form.

Let me introduce you to SKOSsy. I will explain what this web service can do for you:

SKOSsy generates SKOS based thesauri in German or in English for a domain you are interested in. Not any domain but nearly any: SKOSsy extracts data from DBpedia, so it can cover anything which is in DBpedia. Thus, SKOSsy works well whenever a first seed thesaurus should be generated for a certain organisation or project. If you load the automatically generated thesaurus into an editor like PoolParty Thesaurus Manager (PPT) you can start to enrich the knowledge model by additional concepts, relations and links to other LOD sources. But you don´t have to start in the open countryside with your thesaurus project.

Let me give you an example: Imagine you are working for a company which is an international plant builder and you would like to index several thousands of documents the “semantic way”. You have to walk through the following steps:

1. Identify proper categories in Wikipedia/DBpedia which describe best what your business or your domain is all about. Those categories should contain pages / resources which are related to the documents you would like to index. For example: http://dbpedia.org/resource/Category:Metalworking or http://dbpedia.org/resource/Category:Industrial_automation
2. After you have selected proper categories SKOSsy will traverse DBpedia for you and collect all resources, their hierarchical and non-hierarchical relations, alternative labels, definitions and other properties and put them together as a valid SKOS thesaurus; this step will last a couple of minutes. (Find the resulting vocabulary here)
3. Load the resulting thesaurus into PPT, explore it, improve it and enrich it with additional facts.
4. After you´re done you can generate a tailor-made text extractor by using PoolParty Extractor (PPX) which is the second component of PoolParty product family
5. With PPX and its extraction model especially curated for your special use case you can extract named entities from your documents automatically and index your documents in a meaningful way.
6. After a few seconds your semantic search engine is ready to be used. PoolParty Semantic Search (PPS) which is the third PoolParty component will offer some nice facilities like categorized auto-complete, faceted search, content recommendation (similarity search) and smart search suggestions to ease your life as a knowledge worker.

We have constantly discussed the application of thesauri and other knowledge models to improve search over the last years. Many people understood straight away why thesaurus based search is most often much better than search algorithms purely based on statistics. Of course the big contra always was, “the costs are too high to establish a “good-enough” thesaurus or even a “high-quality” one”.

With SKOSsy in place those kinds of arguments become weaker and weaker. To sum up,

• SKOSsy makes heavy use of Linked Data sources, especially DBpedia
• SKOSsy can generate SKOS thesauri for virtually any domain within a few minutes
• Such thesauri can be improved, curated and extended to one´s individual needs but they serve usually as “good-enough” knowledge models for any semantic search application you like
• SKOSsy based semantic search usually outperform search algorithms based on statistics since they contain high-quality information about relations, labels and disambiguation
• SKOSsy works perfectly together with PoolParty product family

If you are interested in the results produced by SKOSsy, just send us a short note about your domain or your project and we will send you an invitation as beta-tester or prepare a demo for you.

Related articles

• Geological Survey Austria launches thesaurus project (semantic-web.at)
• PoolParty 3.0 and its all new Linked Data framework (semantic-web.at)
• PoolParty DemoZone Content Extractor Semantic Search Thesaurus Manager (poolparty.punkt.at)
• Query DBpedia for multiple keywords (stackoverflow.com)

Can you please tell us about the aims and purpose of your tutorial and the importance of incorporating linguistic information in the Semantic Web?

With the continuing growth of linked data and semantic technologies the incorporation of linguistic descriptions into Semantic Web resources has become a challenging issue. The integration of linguistic information especially on a multilingual level could greatly benefit Natural Language Processing (NLP) applications. Furthermore, the continuing growth of ontologies for semantic modeling and the use of terminological resources to add human language descriptions has raised the issue of how to add linguistic information to ontologies and linked data vocabularies and to represent models of lexical and terminological information in a way which is compatible with Semantic Web standards. Prominent examples here are, for instance, multilingual language tags in RDF Schema or SKOS’s success in bringing terminological information to the Semantic Web.

In the Tutorial we would like to discuss trends and novel models such as Lemon – the lexicon model for ontologies – to show possible future directions. The tutorial is targeted at researchers and practitioners interested in learning how to enrich ontologies with linguistic information in one or several natural languages and NLP tool developers interested in understanding how Semantic Web resources can be leveraged fro NLP. There will be two hands-on sessions in this tutorial.

Why did you choose to use PoolParty thesaurus management system in your tutorial?

To create terminology models on the web there are only few tools available which are often very technical and not straightforward to use for non-experts. We found that PoolParty in contrast to other SKOS editors has an attractive and usable interface. In addition the web based interface was preferable, as it did not require the participants to download software, the immediate publishing of linked data is more compatible with linked data principles and the tool has similarities to our own tools for working with lemon.

Thank you for this interview!

Providers of first generation semantic technologies calculate rather basic “semantic networks” by co-occurency analysis which results sometimes in  disappointing results. Bearing in mind that Google just bought a company (“Google buys Metaweb“) which has been working on one of the largest knowledge bases in the world, we could assume that some of the last miles towards a semantic search engine can be achieved by applying thesauri or other structured knowledge bases.

A demo application was recently developed by PoolParty team where one can find out how thesauri will improve search results on top of second generation semantic technologies. With PoolParty SKOS based controlled vocabularies can be managed and also can be enriched with linked data. PoolParty Tag & Content Recommender analyzes virtually any text or website to recommend corresponding tags, concepts from (in this case) STW (Standard Thesaurus für Wirtschaft), DBpedia and respective articles from Wikipedia.

STW which was developed by the German National Library of Economics (ZBW) provides vocabulary on any economic subject: about 6,000 standardized subject headings and about 18,000 entry terms to support individual keywords.

This background knowledge is used in this demo app to improve the search for similar documents dramatically:

Similarity between two documents can be calculated not only on a key-phrase basis but also on a rather conceptual basis. Even if two documents do not have one single word or phrase in common they can be identified as “similar documents”.

This can be achieved because thousands of important relations between economic subjects are represented in the domain specific thesaurus. Thus, in this special case best results are achieved with documents from economics (for instance from Econstor) but of course for other recommender systems thesauri from other domains can be used instead of STW.

Nevertheless, also this approach can be improved and this development is underway: SKOS thesauri enriched with Linked Data do an even better job. This kind of third generation semantic technologies are currently developed by LASSO project and LOD2 project, two innovative projects in the area of linked data and the semantic web.

Each of those services follow a slightly different approach, but in a nutshell: They all offer an API to provide “similarity search” around social media or also to enhance enterprise information management.

Like a magic bullet those services offer a relief from information overflow and seem to become kind of a “semantic web killer application“.

If you´re familiar with one or many of those services, drop a comment and let us know, what you´ve been experienced so far, or also if you can think of any applications or further developments you would like to see around these kind of services.

If you are not familiar with this stuff, for a quick demo go to

• Zemanta´s Demo zone,
• Calais Viewer or see the
• textwise widget below.

The widget uses text from this blog to calculate similar stuff from the web.

Related articles by Zemanta

• semantic technology conference 2009 (web2express.org)
• Zemanta: semweb at work for your blog (squio.nl)

In recent days – here at Semantic Web Company – we have had a lot of discussions on how the future of the Semantic Web (name it Web3.0 if you like) will develop. Several stakeholders on the future of the Semantic Web see already, that also a potential danger will come along with the technical realisation of the web3.0: This is the present possibility to create applications and mashups with semantic technologies that are a real drain on privacy and information ethics. Without an underpinning discussion about the ethical framework within technolgies like linked data, text-mining, biometric-systems and geo-systems in combination with the web of data, the whole domain is in danger to be doomed like genetic engineering some years ago.

It’s crucial for the public opinion on the Semantic Web, to adress the immanent risks regarding privacy and ethics. In this context I’ll see also Tim Berners-Lee‘s statement yesterday: “W3C wants to help make sure data use is appropriate,” he said. Berners-Lee, who is director of W3C, said in an interview on Wednesday that the teams working on the Semantic Web project are making sure that privacy principles are included in its architecture: “The Semantic Web project is developing systems which will answer where data came from and where it’s going to — the system will be architectured for a set of appropriate uses.”

Maybe it’s an important step in keeping the further development of Semantic Web trusty in the eyes of public opinion, that the W3C has privacy and information ethics on their agenda and persons like Berners-Lee stand with their reputation for it. But it is also crucial to build this awareness on the corporate side. Only if everyone within the domain follows a common ethic understanding we have a public opinion, which is on the future potential of the Semantic Web, and not in fear of the same.

Previous releases of Thomson Reuter’s Open Calais web service already produced promising results by extracting named entities, facts and events from user submitted contet – especially news articles. Now these extracted concepts come with an URI and are linked into the LOD cloud – specifically to DBpedia, Freebase, Musicbrainz, CIA world fact book and others.

On this occasion Tom Tague, vice president of the Calais creators ClearForest, answered questions the Semantic Web Company had about the goals of Open Calais. 

The latest release of Open Calais produces metadata conforming to linked data principles. You provide this great service free to everyone via your web service.
What led to that decision, which benefits are there for Thomson Reuters?

Thomson Reuters has the largest trusted content sources in the world – but we don’t have all the content in the world. We believe that the world is going to want to integrate highly managed and trustworthy content assets such as those provided by Thomson Reuters with the low latency, highly diverse content exploding on the web. Fundamentally what we’re trying to achieve is nearly effortless interoperability of content between any two partners – Calais enables this by extracting the semantic metadata buried in your content but then takes it a step further. By linking those semantic elements to the Linked Data cloud we are setting the stage for the dramatic enhancement of any content source – and we hope that many will choose Thomson Reuters as one of the methods for enhancing that content.

It seems with Open Calais you use a hybrid business model, which integrates end users in a form of enterprise collaboration into value creation.
Do you think such a business model is viable during the long run and what are your experience so far?

As of right now Calais isn’t truly a “Business”.  It’s a strategic initiative that’s setting at least a piece of the stage for the Linked Content Economy. Our goal is to understand how this new content economy is going to involve and to make certain that we have a leadership position as it moves from a concept to reality.

Apart from the thousands of users submitting content to Open Calais, there is also a community of developers making their own applications around your core app. How important are the social dynamics of the Open Source community for the success of Open Calais?

Extraordinarily important. Calais is a web service – which means it’s relevant to about 0.0001% of the population. We are absolutely reliant on the creativity, energy and domain expertise of our developer community to translate Calais from a technology to an end-user relevant capability. And – as a user-driven project we also rely on our developers and users to give us feedback on what they like, what they don’t and where they think we should head.
What are your plans regarding to offering your service in German?

We hope to get there in 2009. We’ve released basic French and are gearing up for additional languages in the coming year.

Thank you, Tom, for your answers! We look forward to more applications like Semantic Proxy and Linked Facts that demonstrate the great protential of the Calais engine.

Jinni provides recommendations, answering a free given search. Based on semantic technologies Jinni uses Natural Language Processing on plot, mood, style, setting, soundtrack and more in combination with an ontology, created by film professionals (like Jinni says). When it launched in December, Jinni had 10,000 movie, TV and video titles.

In Jinni you don’t need to know about exact title, actor, director, place or year of production to get an result, you can enter simply a phrase describing the mood, genre or place the movie is about, and you will guided through a facilitated search to narrow your search and get at the end what you want. Or alternative, if you search for a movie and you have only a vague idea of the plot, you can formulate a plot’s description in free phrasing. As it also offers APIs for Internet and TV content providers you can make your way direct to an online store to download or purchase the movie.

Another idea how to develop business orientated semantic web services comes with BooRah. BooRah is a service targeting restaurant owners to provide them reports of positive and negative reviews of food, service and ambiance at their restaurants. For that the service monitors negative and positive trends across hundreds of online review sites. Now restaurant owners can subscribe to receive a PDF of their monthly reports for an introductory price of $15 and a regular price of $25 per month. This PDFs came with charts, trends, rankings, summaries and some quotes from users, month by month. The reports may enable those restaurant owners to react and improve their services in the specific field. A simple but straight forward way to make money with semantic technologies

Related articles

• BooRah Now Selling Semantic Yelp, CitySearch Report Cards
• Jinni: Wants to be Pandora for Movies

First off: What is Information Extraction? In natural language processing, “information extraction (IE) is a type of information retrieval whose goal is to automatically extract structured information, i.e. categorized and contextually and semantically well-defined data from a certain domain, from unstructured machine-readable documents” (Wikipedia). Marek and Petr’s vision for using IE in KiWi is to support the user in the creation of semantic annotations.

The image above illustrates their vision: If a user, for instance, enters the text “Hello, I am the best expert in Java around the Sun” into the content editor, structured information is extracted, analyzed on the fly and returned as suggestions. Through the application of reasoning on existing annotations and on further information that is available on the system – e.g. relevant domain ontologies, but also information about the user himself – the system will be able to infer new statements: E.g. the system will be able to infer that Bill Rodgers has Java programming skills, even though this information has never been explicitly stated in the knowledge base.

The goals of LMU’s contribution to KIWI are:

• to develop a rule-based language that can be used by wiki users to specify queries and derivation rules, ideally in a simple and intuitive way
• to develop a reason maintenance component for this language that gives users an opportunity to understand why derivations exist and that allows for versioning of updates of the knowledge base