Andreas Blumauer

Automatic text analytics using DBpedia and PoolParty – A Live Demo

Let me show you which steps have to be taken to generate a high-quality text mining application, ready to be used to annotate and to categorize any kind of text or documents covering nearly any domain. With our approach of thesaurus based text mining your documents can also be linked to the world of linked (open) data; enrich your documents with data from the LOD cloud!

Step 1. Generate a thesaurus by using a linked data source like DBpedia

As recently reported SWC has developed a tool called SKOSsy which can be used to extract seed thesauri from DBpedia. In our example I will generate a knowledge model describing the domain of “digital photography“. This step took around 15 minutes.

Step 2. Load the thesaurus into PoolParty and improve it to your needs

After the seed thesaurus has been loaded into PoolParty Thesaurus Manager you have many possibilities to enhance the knowledge model further: Add more categories, synonyms, relations etc. In this example I use the seed-thesaurus without any further improvements. This step took approximately 2 minutes.

Step 3. Generate an automatic text extractor on top of your thesaurus

This step took a couple of seconds and ended up in having generated a fast and reliable text mining application on top of PoolParty Extractor, ready to be used to enrich your documents with data from the LOD cloud.

You can try it out here: PPX Live-Demo

To try the extractor on your own, please take a look at the image above which shows a proper configuration, you have to insert the following UUID in the form: d35d4ddb-adc3-4ea5-b027-deacac03e391

Since our example is all about ‘digital photography’, we recommend to use text samples (or some fragments) like these ones to test the quality of PPX based text analytics:

Let us know what you think about this straight-forward approach and your opinion about the quality of the results. We believe that thesaurus based text mining is in many cases an alternative to some other approaches, especially if you want to to enrich your content with information from the upcoming web of data.

Of course we would be happy to generate other demos in the areas of your interest! Just get in contact with us by using our contact form.

Andreas Blumauer

Linked Open Data: The Essentials – A quick start guide for decision makers

Together with REEEP (Renewable Energy and Energy Efficiency Partnership) the Semantic Web Company (SWC) has composed a fundamental publication on the topic of Linked Open Data.

Linked Open Data: The Essentials provides answers to the following key questions:

  • What do the terms Open Data, Open Government Data and Linked Open Data actually mean, and what are the differences between them?
  • What do I need to take into account in developing a LOD strategy?
  • What does my organisation need to do technically in order to open up and publish its datasets?
  • How can I make sure the data is accessible and digestible for others?
  • How can I add value to my own data sets by consuming LOD from others?
  • What can be learned existing best practices?
  • What are the key potentials of sharing and consuming open datasets?

Read more about this publication and find out how to obtain a copy.

Andreas Blumauer

Experiences from teaching Linked Data

Dr. Bernhard Haslhofer works as instructor on Web Information Systems at Cornell Information Science. Just recently he gave a course which examined technologies for building data-centric information systems on the World Wide Web. Semantic Web Company (SWC) had the opportunity to talk with Dr. Haslhofer to examine the question “How to teach Linked Data?“.

SWC: Bernhard, you have been working on the Semantic Web and Linked Data for years now. What is the first lesson you usually give when you try to explain the “Semantic Web”?

Maybe I should first clarify that the course I am co-teaching is not a Semantic Web course. The course is about data-centric Web information systems in general and we spent some classes talking about Linked Data and Semantic Web technologies. We start explaining the origins and the fundamental architectural principles of the World Wide Web and then focus on the data-centric aspects of the Web.

“instead of building isolated repository-centric APIs we could also build a globally connected data graph

After introducing various data exchange formats (XML, JSON & co.) we teach how Web APIs work, and discuss the design principles of RESTful Web Services. Then the conceptual transition to Linked Data is just a small step, because we can argue that instead of building isolated repository-centric APIs we could also build a globally connected data graph, which is based on a uniform data model and can be traversed and queried using SPARQL.

“DBpedia and all the other existing Linked Data projects and tools that came up in recent years really help in explaining and illustrating how things work”

So, I am somehow approaching the “Semantic Web” bottom-up and concentrate on the “visible” parts of the “Semantic Web” vision. DBpedia and all the other existing Linked Data projects and tools that came up in recent years really help in explaining and illustrating how things work. And last but not least, schema.org and the design of the Facebook Open Graph protocol also show the growing importance of having structured data on the Web.

SWC: At least for non-technicians “Linked Data” sounds very technical. Antoine de Saint-Exupery said: “If you want to build a ship, don’t drum up people to collect wood and don’t assign them tasks and work, but rather teach them to long for the endless immensity of the sea.” Is there an “endless immensity of the sea” you try to bring in as well?

If you can access and combine data from the Web you can answer interesting questions and discover previously unknown relationships between things. We thought the best way to learn about Linked Data is to implement simple demo applications. So we asked the students to think about uses cases that bring some benefit for end users and require data from several Web sources to answer certain questions.

“I think it became clear what it means to work with easily accessible structured Web data opposed to working with unstructured data”

One group developed a service which connects safety records with public transport information. Now users can now easily choose the “safest” bus connection between from and to New York City and other cities. Another group combined public school district information with geographic data, which now allows parents to view statistical information about school districts in New York State by using apps like Google Earth. There are many more examples, but most importantly, I think it became clear what it means to work with easily accessible structured Web data opposed to working with unstructured data.

SWC: Instructing how to use the Semantic Web is not only a matter of slide-decks. It is rather a question of concrete use cases in combination with tool skills. What kind of tool skills should students of information sciences acquire to your opinion?

Collecting and making sense out of data is a common scholarly practice in many research areas and the Web is becoming, or is already, the primary medium for publishing and distributing results. I believe that making data accessible as part of a some research activity will become increasingly important in future and the Web will probably be infrastructure for doing this.

So I think that a student who is working with data should at least know (i) how to retrieve and (ii) how to publish data on the Web in way that others can easily discover, access, and use their data. Linked Data is one possible technical approach for doing that.

SWC: As a European who is teaching and working in the U.S., how do you perceive the different approaches between those two systems when it comes to transfer complex fields of knowledge like the semantic web from universities to business environments?

From the experiences I have made in my previous and current working environments I can only tell that the relations between businesses and universities seem to be tighter in the US. I don’t necessarily mean “formal” bounds between institutions but rather informal relations between people, who understand complex fields of knowledge, both in the academia and in business.

“I assume transferring knowledge between two proxies who speak the same ‘language’ makes it a lot easier”

PhD students, for instance, often work in business over the summer and/or continue their career in the research department of some company. Some continue their cooperation with their former professors and academic colleagues and I assume transferring knowledge between two proxies who speak the same “language” makes it a lot easier.

SWC: What are the most important things which are still missing to make linked data technologies an integral part of enterprise information systems?

Quite often I hear the complaint that major database vendors still don’t provide satisfactory RDF support in their products. I don’t think this is a necessary precondition for implementing Linked Data but for some institutions this seems to be very important.

Many thanks!

Enhanced by Zemanta
Andreas Blumauer

Introducing SKOSsy – generate thesauri on the fly!

Imagine you could generate any thesaurus you would like for nearly any knowledge domain you can think of with quite a good quality! Sounds impossible? Reminds you of all the promises made by text mining software which generates “semantic nets” from scratch?

Let me introduce you to SKOSsy. I will explain what this web service can do for you:

SKOSsy generates SKOS based thesauri in German or in English for a domain you are interested in. Not any domain but nearly any: SKOSsy extracts data from DBpedia, so it can cover anything which is in DBpedia. Thus, SKOSsy works well whenever a first seed thesaurus should be generated for a certain organisation or project. If you load the automatically generated thesaurus into an editor like PoolParty Thesaurus Manager (PPT) you can start to enrich the knowledge model by additional concepts, relations and links to other LOD sources. But you don´t have to start in the open countryside with your thesaurus project.

Let me give you an example: Imagine you are working for a company which is an international plant builder and you would like to index several thousands of documents the “semantic way”. You have to walk through the following steps:

  1. Identify proper categories in Wikipedia/DBpedia which describe best what your business or your domain is all about. Those categories should contain pages / resources which are related to the documents you would like to index. For example: http://dbpedia.org/resource/Category:Metalworking or http://dbpedia.org/resource/Category:Industrial_automation
  2. After you have selected proper categories SKOSsy will traverse DBpedia for you and collect all resources, their hierarchical and non-hierarchical relations, alternative labels, definitions and other properties and put them together as a valid SKOS thesaurus; this step will last a couple of minutes. (Find the resulting vocabulary here)
  3. Load the resulting thesaurus into PPT, explore it, improve it and enrich it with additional facts.
  4. After you´re done you can generate a tailor-made text extractor by using PoolParty Extractor (PPX) which is the second component of PoolParty product family
  5. With PPX and its extraction model especially curated for your special use case you can extract named entities from your documents automatically and index your documents in a meaningful way.
  6. After a few seconds your semantic search engine is ready to be used. PoolParty Semantic Search (PPS) which is the third PoolParty component will offer some nice facilities like categorized auto-complete, faceted search, content recommendation (similarity search) and smart search suggestions to ease your life as a knowledge worker.

We have constantly discussed the application of thesauri and other knowledge models to improve search over the last years. Many people understood straight away why thesaurus based search is most often much better than search algorithms purely based on statistics. Of course the big contra always was, “the costs are too high to establish a “good-enough” thesaurus or even a “high-quality” one”.

With SKOSsy in place those kinds of arguments become weaker and weaker. To sum up,

  • SKOSsy makes heavy use of Linked Data sources, especially DBpedia
  • SKOSsy can generate SKOS thesauri for virtually any domain within a few minutes
  • Such thesauri can be improved, curated and extended to one´s individual needs but they serve usually as “good-enough” knowledge models for any semantic search application you like
  • SKOSsy based semantic search usually outperform search algorithms based on statistics since they contain high-quality information about relations, labels and disambiguation
  • SKOSsy works perfectly together with PoolParty product family

If you are interested in the results produced by SKOSsy, just send us a short note about your domain or your project and we will send you an invitation as beta-tester or prepare a demo for you.

Enhanced by Zemanta