KiWi, the Open Source development platform for building Semantic Social Media Applications, offers features required for Social Media applications such as versioning, (semantic) tagging, rich text editing, easy linking, rating and commenting, as well as advanced “smart” services such as recommendations, rule-based reasoning, information extraction, intelligent search and querying, a sophisticated social reputation system, vocabulary management, and rich visualisation.

Marco, you recently initiated a discussion in a Google Group on the difficulty to change Semantic Web standards. What was the background of the discussion? Where do you perceive a need for action?

It’s not so much about changing this existing standards but the challenge to bring them into the world of practitioners and standards developers. The language used in W3C recommendations quite frequently requires advanced topic knowledge and familiarity with the jargon of the discussion about the respective technologies. I recently discussed this with a senior standards maven at the W3C and got the answer that the recommendations can’t be changed retrospectively and that they are intended to be used primarily by vendors for implementation purposes.

Well this might be the case but I also got the impression that Tim Berners-Lee objective for the W3C is primarily to meet the needs of a larger community. And the W3C took this into account for most of the Semantic Web recommendations in the past. Something I still find amazing is the fact that the work process at the W3C is partially and the recommendations are entirely publicly accessible. Though we definitely still need more and better tools to work with semantic web data, higher quality documentation and last but not least more user adoption on the web.

Critics of the Semantic Web often refer to the slow uptake of Semantic Web standards by industry. Is standards adoption actually a valid and sufficient metric to evaluate the maturity of a standard? What would be needed to accelerate the uptake?

I think we might see a similar scenario to the uptake of HTML in the early 90s, a relatively small number of technology mavens will pave the way towards making the Semantic Web more attractive as a technology solution for a wide range of applications and will successfully publish open data before we see business application developers make use of Semantic Web standards.

The availability of trustable and quality approved RDF data is crucial for the success of the Semantic Web. Given the fact that the aggregation business on the WWW is highly concentrated the corresponding formula is simple: If Google just consumes but does not give back RDF the Semantic Web won’t scale. Do you agree?

Yes and no. Yes we need better and more semantic data on the Web, but we will also need better ways to deal with trust in a lightweight and web friendly fashion. I currently see a number of semi automated approaches emerging  that could scale on the web. An example are distributed user based recommendation systems to validate authenticity, open Wikipedia style community evaluation and content curation a la freebase. Increased public accountability for data producers might be an interesting venue as well. In regards to Google I’d say web search engines will go where the web goes. A problem I might see arising is that web search engines will initially develop their own standards to deal with the emerging Semantic Web and confuse users on the web or might pursue a time consuming power play with the W3C. I see a little bit of that in the current discussion in the HTML 5 working group.

As we know from social sciences technological standards are necessary but always incomplete and unsatisfactory. From a standards design and outreach perspective: What would it need to make the Semantic Web flourish?

I’m not sure if we really know all that much about the laws of innovation and the evolution of technology standards at this point. If we draw from the short experience with the World Wide Web I would come to the conclusion that innovation takes place in small to medium size teams that pursue an independent vision of how services should be delivered and how the technology should be designed. In addition Tim Berners-Lee’s encourages the production of lots and lots of data to bootstrap the Semantic Web and create a pull for services in the industry. And indeed we really see some traction for example with the Linked Open Data and Open Government initiatives. It’s definitely an exciting time to be on the Semantic Web!

About Marco Neumann

Marco Neumann is an Information Scientist and CEO of KONA a consulting and technology service company based in New York City. KONA provides semantic technologies to businesses solutions and adds value to products and services in a highly networked economy. In addition Marco currently acts as an Invited Expert to the W3C on the HTML 5 working group and is the director of the global semantic social network lotico.com.

A new QuadStore which combines the best of two worlds (Lucene/Fulltext search engines & TripleStores/RDF/SPARQL) is out and can be evaluated online.

TuQs offers the following feature:

• SAIL accessible
• True QuadStore with GraphSupport
• HighSpeed regex SPARQL filters
• Userrights on TripleBasis
• Extendable to a QuintStore (or more generally to an n-Store)
• Cachable SPARQL Queries for further speed improvement
• Clusterable
• Federationable
• FullTextSearchable

Some queries are really complex and high-speed, e.g.:

SELECT ?s ?o
WHERE {
?s <http://www.w3.org/2004/02/skos/core#definition> ?o .
?o <http://www.turnguard.com/tuqs/function#BooleanTerm> ‘Computer AND (java* OR HTML)’
}

The best starting point to find out, what´s the speciality of TuQS is here: Just click the sample queries on the right side and see how fast they perform even on very simple hardware.

Next steps: The developer of TuQS, Jürgen Jakobitsch (aka Turnguard), is currently working on SAIL inferencing.

Purpose

Poolparty was conceived to facilitate various applications like

• Semantic search engines
• Recommender systems (similarity search)
• Corporate bookmarking
• Annotation- & tag recommender systems
• Autocomplete services and facetted browsing.

These use cases can be either achieved by using PoolParty stand-alone or by integrating it with existing Enterprise Search Engines and Document Management Systems or Enterprise Wikis.

Thesaurus Management

PoolParty is aiming to be easy to use for people without a strong Semantic Web background or special technical skills. The GUI is entirely web-based and utilizes AJAX so the user can e.g. quickly merge two concepts via drag & drop. An overview over the thesaurus can be gained with a tree or a graph view on the concepts.

PoolParty also helps to semi-automatically add concepts to a thesaurus as it can be used to analyse documents (e.g. web pages or PDF files) relevant to a thesaurus’ domain in order to glean candidate terms. This is done by the key-phrase extractor of KEA. The extracted terms can be selected by the user, thereby becoming “free concepts” which later can be integrated into the thesaurus, turning them into “approved concepts”.

Documents can be searched in various ways – either by keyword search in the full text, by searching for their tags or by semantic search and similarity search. The latter takes not only a concept’s preferred label into account, but also its synonyms and the labels of its related concepts are considered in the search. The user might manually remove query terms used in semantic search. Boost values for the various relations considered in semantic search may also be adjusted. In the same way the recommendation mechanism for document similarity calculation works.

PoolParty by default also publishes a Semantic Wiki version of its thesauri, which provides an alternative way to browse and edit concepts. Through this feature anyone can get read access to a thesaurus, and optionally also edit, add or delete labels of concepts. Search and autocomplete functions are available here as well. The Wiki’s XHTML source is also enriched with RDFa, thereby exposing all RDF metadata associated with a concept to be picked up by RDF search engines and crawlers. (See two examples: Cocktail thesaurus &  Standard Thesaurus for Economics)

PoolParty also supports the import of thesauri in SKOS (including several consistency checks) or Zthes format. Those functionalities can also be consumed as stand-alone web services via PoolParty SKOS Services. Additionaly, lists of concepts and their labels can also be imported via CSV files.

Linked (Open) Data

PoolParty not only publishes its thesauri as Linked Open Data (in addition to a SPARQL endpoint), but it also consumes LOD in order to expand thesauri with information from LOD sources.

Concepts in the thesaurus can be linked to e.g. DBpedia  via a service like Georgi Kobilarov‘s DBpedia lookup service, which takes the label of a concept and returns possible matching candidates. The system suggests relevant resources from DBpedia and the user can select the one that matches the concept from his thesaurus, thereby creating a skos:exactMatch relation between the concept URI in PoolParty and the DBpedia URI. The same approach can be used to link to other SKOS thesauri available as Linked Data.

Other triples can also be retrieved from the target data source, e.g. the DBpedia abstract can become a skos:definition and geographical coordinates can be imported and be used to display the location of a concept on the map, where appropriate. The DBpedia category information may also be used to retrieve additional concepts of that category as siblings of the concept in focus, in order to populate the thesaurus.

PoolParty is capable of importing a SKOS thesaurus from a Linked Data server, and may also receive updates to thesauri imported this way. This feature has been implemented in the course of the KiWi  project funded by the European Commission. KiWi also contains SKOS thesauri and exposes them as LOD. Both systems can read a thesaurus via the other’s LOD interfaces and may write it to their own store. This is facilitated by special Linked Data URIs that return e.g. all the top-concepts of a thesaurus, with pointers to the URIs of their narrower concepts, which allow other systems to retrieve a complete thesaurus through iterative dereferencing of concept URIs.

Additionally KiWi and PoolParty publish lists of concepts created, modified, merged or deleted within user specified time-frames. With this information the systems can learn about updates to one of their thesauri in an external system. They then can compare the versions of concepts in both stores and may write according updates to their own store.

This means each system decides autonomously which data it accepts and there is no risk of a system pushing data that might lead to inconsistencies into an external store. Data transfer and communication are achieved using REST/HTTP, no other protocols or middleware are necessary. Also no rights management for each external systems is needed, which otherwise would have to be configured separately for each source.

Technology

The software is written in Java and utilizes the SAIL API, so it can be used with various triple stores. The thesaurus management itself (viewing, creating and editing SKOS concepts and their relationships) can be done in an AJAX Frontend based on Yahoo User Interface (YUI). Editing of labels can alternatively be done in a Wiki style HTML frontend. For key-phrase extraction from documents PoolParty uses a modified version of the KEA 5 API, which is extended for the use of controlled vocabularies stored in a SAIL Repository (this module is available under GNU GPL). The analysed documents can be stored and indexed in Lucene/Solr or any other (enterprise) search system along with extracted and semantically related concepts.

We started experimenting with the technical infrastructure needed and created some proof-of-concept applications. Part of this work was enabling Linked Data access for the front-end infrastructure we used, Liferay portal. We decided on the appropriate vocabularies for the type of content we wanted to publish (FOAF, SIOC and MOAT mainly), delved on the internals of Liferay and used D2R to map its relational database to the vocabularies of choice, also using techniques to improve performance as much as possible. Since Liferay itself is also based on the notion of communities, we thought our work would be more widely applicable and useful, so we chose to submit it for review at the Triplification Challenge and make it available to the community as open source software. Our applications have gradually matured and are about to be deployed in our commercial projects, while at the same time we are now making the Liferay Linked Data Module available as a Sourceforge project and we are working with Liferay management in order to disseminate this effort to the community and also include it in a future release of the software.

Read the full interview here.

Maybe you have noticed it already; today in the morning something new appeared at Google’s search engine interface: A bunch of corresponding search-suggestions based on your search query. Google spoke about this enhancement:

Starting today, we’re deploying a new technology that can better understand associations and concepts related to your search, and one of its first applications lets us offer you even more useful related searches (the terms found at the bottom, and sometimes at the top, of the search results page).

I tried it. So, if you type in “time travel” you also get search proposals like “theory of relativity time travel” or “wormhole time travel”. Google annouced, that the service is available in various languages. The direct test with German is a little disillusioning: Searching for “zeit reise” (which is the same concept as above, in german) leads to alternative searches like “reisen 50er jahren” (travel 50ies) and “reisen im mittelalter” (travel in the medieval).

Even if this semantic-like extension of the basis search function still needs some tuning, the point is getting clearer: Also Google is doing developments to get more meaningful results into their search algorithms. And parts of the semantic methodology are finding their way into mainstream services like search engines – as we have seen with Wolfram Alpha some days ago. So keep your eyes open – maybe next morning you’ll find another piece of the semantic puzzle embedded into one of your favorite web-apps.

It is thus high time to tell you about them. Let’s start with the new layout, introduced in December. It’s much closer to the Web 2.0 look & feel, with pastel colors and rounded corners. The navigation has become a bit more consistent, and you can now select if you want to see both bookmarks and publications, or just one of the two. BibSonomy is also available in German now. Most other extensions of BibSonomy are about integration with other systems. The most useful are:

• GoogleSonomy is a new firefox addon which integrates search results from BibSonomy directly in your Google search. The addon is customizable so that you can decide whether to search in your personal publications and/or bookmarks, or to search over all BibSonomy posts.The extension is available from the Mozilla Addon Page.
• BibSonomy now also allows to export citation information to Zotero. Zotero is a Firefox extension, which helps you to collect, manage and cite publications locally in your browser. The other way around is not fully automized yet. However, there is a copy and paste workaround.
• Bloggers who are using WordPress can integrate data from BibSonomy into their posts – for instance your tag cloud, or your last three publications (or all of them). Conversely, your blog posts will (almost) automatically be published on BibSonomy. A more general way of including BibSonomy content into your system is BibSonomy’s JSON feed. JSON (JavaScript Object Notation) is a lightweight data-interchange format, which is now available for all BibSonomy pages.
• As alternative login procedure, BibSonomy now also supports OpenID, which is an open, decentralized standard, allowing users to log onto many different services on the web using the same identity identification (single sign-on). This kind of authentication is provided by a growing number of websites, including large ones like AOL, Google, Microsoft, MySpace, Yahoo and many others. You may even have an OpenID without knowing so, e.g. when you have a Flickr account. Why not using it for logging in to BibSonomy as well?
• The family of scrapers for automatically extracting references from digital libraries or publishers’ websites has been extended, allowing you to store publication metadata automatically from over 60 sites. The scraping service can be used independently from BibSonomy for other purposes by everyone needing access to bibliographic metadata.

If you want to learn more about these features, visit the BibSonomy blog. Last but not least there exists a new BibSonomy developer site. It provides access to some of the BibSonomy modules. All source code is released under GPL LGPL licenses. If you want to experiment with the code, have a look!

Just like the overall organization of the project meeting was very good, so Sebastian Dietzold, Sebastian Hellmann, Michael Martin and Jörg Unbehauen did a real good job at putting the ideas behind key concepts of the semantic web across in several introductory SemWeb presentations. Their talks about various technologies from the semantic web stack like URIs, RDF and its serialisations, RDFS, SPARQL and some related tools were well suited to bring people who are relatively new to the semantic web up to speed. Links to the presentation slides can be found at the project page in the coming days.

Later Jens Lehmann outlined the new things OWL 2 brings, e. g. profiles, which are subsets of OWL 2 and which provide different degrees of expressivity and reasoning efficiency.

The last day started with Sören Auer’s presentation of their semantic wiki OpenResearch, a site where information on conferences, journals and scientists is pooled. OpenResearch is built with Semantic MediaWiki (SMW), just like our Social Semantic Web wiki.

While SMW is a very useful tool as it lowers the entry barriers for using semantic wikis, Sören also pointed out  that in comparison OntoWiki provides some important features that SMW doesn’t have:

• SMW doesn’t use SPARQL for its queries, but a less powerful custom query language, whereas OntoWiki has full SPARQL support.
• OntoWiki’s UI has many widgets that support the user when entering data or new properties on a page (e. g. there is an autocomplete feature for suggesting properties)
• With SMW changes to the wiki’s semantic structure often entail manual changes to many, many pages. With OntoWiki it is easy to e.g. change poperties at any time.

For the new version of OntoWiki Sören and his team use the Zend framework and develop the Erfurt API to store and access RDF data. The Erfurt API supports SPARQL, versioning, caching and RDF based authentification/access control. It abstracts different stores using the adapter pattern, so it can be used with Virtuoso and any other store which has an interface provided by Zend_Db (MySQL, Oracle, PostgreSQL, etc.) plus they are working on an interface for Redland. Find the slides for Philipp Frischmuth’s Erfurt API presentation here, the API documentation here and Norman Heino’s Zend & OntoWiki Application Framework presentation here.

Julian Jöris demonstrated how Selenium is used for acceptance testing. This is a very promising testing framework for web applications, where one can e.g. record interactions with different browsers and automatically run them as tests. Selenium has a Firefox extension to record macros and is integrated with PHPUnit.

Finally we had a very good discussion about our conX-OntoWiki integration use case and application ideas, so we left Leipzig with a pleasant anticipation of the coming co-operation in the project.

In the first presentation today, Sebastian Schaffert from Salzburg Research gave us a tour of two different configurations of the KiWi system. The KiWi core system is oriented towards content versatility, meaning that content items can be displayed and used in various contexts and configurations. As a service to the user, KiWi uses Javascript-based WYSIWYG Editor TinyMCE enhanced with a few home-grown plug-ins which, for instance, make it easier to set links to other wiki pages. Memorizing wiki shorthand is sometimes a challenge, so this feature helps getting things done.

Using a different skin and interface, KiWi can take various forms and shapes – even shapes where you might not spot the wiki in it at first glance. TagIT is such an example of an adaptation of the KiWi core system: a geotagging platform targeting youth in Salzburg who can locate, tag and comment on places that matter to them.

Vice versa, KiWi in its wiki incarnation displays a little map, provided a content item is enhanced with geoinformation; technically, the map on the wiki page is an interpretation of a georelated tag (learn more about complex, structured tags proposed by the KiWi Enabling Technologies Work Package in this article: Usage Data Model Day in the KiWi Project).

Take a look at the screenshots below:

It is the same article that is being displayed, in the first example using the classic KiWi interface, in the second example using the TagIT interface with the article appearing as an info page.

This afternoon, we expect to see another configuration of the system, in a presentation about how the system is specifically tailored to the needs of Logica’s “Knowledge Management for Project Management” usecase.

N.B. The system is not yet publicly available, if you have questions, please contact Sebastian Schaffert.

Related articles by Zemanta

• KiWi Joint Work Package Meeting in Prague
• KiWi: Knowledge in a Wiki
• Usage Data Model Day in the KiWi Project
• Not a pipeline, but a graph: software development in the KiWi/Sun usecase

In the Munich Modell, knowledge comes in three states of matter: solid (like ice), liquid (like water) and gas (like water vapor).

“Frozen” knowledge is knowledge in its most tangible, manageable form, for instance the type of verified, expert-endorsed information you would find in an encyclopedia like the Encylopedia Britannica.

“Gaseous” knowledge, on the other hand, is knowledge in its least consolidated form: think for instance of the type of heated debate you might have with folks in a pub, which is arguably the least structured, most uncontrollable, but also the most engaging type of knowledge!

And the “liquid” form of knowledge, eventually, is the common knowledge of day-to-day-life. It’s probably fair to say that it becomes obvious mostly when in the process of changing its state of matter: When it is calibrated against “frozen” or informational knowledge or when it is debated, becomes “gaseous” knowledge that informs action. (If you’d like to know more about the Munich model and are able to read German, you might want to download the original article here – PDF, 365 KB).

When talking about knowledge that is managed, used or, respectively, that evolves online, I think it also makes sense to pay some attention to the type of community that is preferred by particular online tools or environments. The particular flavour of knowledge, in this sense, is simultaneously characterized and shaped by the state of matter of knowledge and the form of the community that applies.

N.B. The following is not an immediate translation of the “Munich model”, but rather a reconceptualization which tries to also consider that different community models (and their implementation through IT) also play a role for the whole spectrum of knowledge management on and with the web (e.g. for online communication and interaction, online publishing and documentation and maintenance of web infrastructures).

Web-Flavour 1: The Blogosphere – gas, gas, gas!

Hmm… sniff it! This is the flavour I like best because it is my flavour. On the blogosphere (and twittersphere), knowledge is exchanged, developed further and evolves almost like in a pub debate… it does have the extra advantage though that you can add links, cite resources and that you get to keep your blog posts (or tweets or equivalents thereof) for later reference or debate. Different people approach blogging differently – the approach I would favour in the context of this definition is a form of blogging that invites dialog in that it allows others to comment and react, and where contributors aren’t anonymous, distant institutions, but are addressable using their personas/identities on the web. As such, contributions are often marked or tinted by the views and personality of that real-life person behind a persona/identity. As a short cut, think of this flavour as the flavour of the social media tag cloud.

Web-Flavour 2: Wikipedia – evolving slowly with the flow

Wouldn’t you agree that Wikipedia is like a sea of knowledge? It is fed by brooks and rivers (in this analogy: for instance the article and contributions that are invited on the Wikipeda Community portal) that make it rise and swell like tidal waves would, but mostly by millions, billions and trillions of tiny drops that trickle in on a daily basis. In comparison to the blogosphere, the world inside wikipedia is a rather neat and orderly one: Titles of pages are unique, and were they aren’t, there are disambiguation pages (like this one) in place. Even though articles are written by real humans (I assume), there is no visible author attached to an article (unless you start developing an interest for Discussion Pages, e.g. this one; most people don’t). Wikipedia is the sea of knowledge we bathe in on a day-to-day-basis without even noticing – just try to remember how many and which Wikipedia pages you have looked at today or this week – can you? Most probably not. It’s the result of a community effort, but it’s not about views and opinions of individuals, but about what we all know together or would know together if we could wire our brains to one another (tired of the Wikipedia examples? Check out Factolex instead, a collaborative, micro-content encyclopedia that allows you to extract and conceptualize bite-sized pieces of knowledge as you go).

This is the flavour I like to have around me every day, because it makes things easier without asking for a huge effort. It’s also the flavour of thesauri and metadata schemata.

Web-Flavour 3: The unfinished structure of ontologies

Flavor three is NOT (as one might expect) the flavour of the Encyclopedia Britannica Online (which is one huge data silo and therefore not relevant for my scope of interests) … instead, I would argue that it’s the flavour of the web’s infrastructures and of knowledge infrastructures like ontologies. Think of the geometry of snow flakes: they all follow rules but none of them is like another. The open world assumption of ontologies also applies to snow flakes – just because you haven’t seen a particular shape doesn’t mean it doesn’t exist! Nobody has the patent for building snowflakes – Wilson Bentley in his famous snowflake shots just captured an expression of rules that are out there, in the world, belonging to the world. Ontologies capture the structure of what, to the best of knowledge and belief, can be said about the world. Anyone can build an ontology, but we prefer to have experts do that job: members of the scientific community which has its own ranking and weeding mechanisms in place.

Flavour three is the flavour of things we wish to be able to rely on on the web, and where we can invest a trust that is much greater than the trust we invest in people. More like the trust we invest in, say, the laws of nature.

So what is the flavour of a Semantic Wiki?

A good mix of flavour two and three, I would argue. A Semantic Wiki is a vessel for the sea of relevant knowledge (relevant for instance for the members of a particular team), but enhances it with the structure of the domain knowledge that applies.

Having said that: A semantic wiki would be much spicier if it also had a bit of the flavour of the blogosphere and social media (i.e. flavour one), as there are tasks where a bit of a debate, a bit of a controversial exchange and the ability to respond to people directly is highly valuable! Just as water, knowledge goes through a cycle of different states of matter, and knowledge is not processed by segregated individuals, but in communities and through networks of people.

Before publishing this, I wanted to get some feedback in particular from KiWi members working on enabling technologies – here is Peter Dolog‘s take; Peter is an Associate Professor in the Information Systems Unit at the Dept. of Computer Science Aalborg University:

I like the distinction and comparison of knowledge to some natural elements like gas, liquids, solids or snowflakes. These give a good metaphor for understanding when talking about different flavours of knowledge. It is also fascinating to see how humans move between these three categories by participating in different social processes or simply by studying these things.

It is, however, a bit more difficult to see how this can be done or supported in the most suitable form on the web or in the intranets of companies. At the same time it seems to me, from the discussions we have had in the KiWi project, that semantic wiki platforms could indeed facilitate this. Wikis naturally provide the social contexts for contributions. Semantic wikis with tags and ontology management seem to be a first step towards a flexible knowledge consolidation infrastructure where one can move easily between these categories; and other technologies such as natural language processing and automated reasoning can help further. Personalization can further provide and adjust views on the knowledge according to preferences.

I am happy that we can study these phenomena in the KiWi project at least to a certain extent and perhaps contribute to this as well. I am confident that this is relevant also for the industry and especially for large distributed companies where externalization of knowledge is a must.

So there is a lesson to be had: When building a knowledge management system using the web, think of the three states of knowledge, but most importantly, also think of the form of community and community processes that are required or preferable to allow future users to really put that knowledge to work – melt it, share it, heat it, debate it, freeze it, keep it, let it evolve!

Image sources on Wikicommons:
Water vapour by Markus Schweiss
Wake at Boelge Stor by Malene Thyssen
Ice Crystals by Petr Dlouhý

Author: Jana Herwig

Related articles by Zemanta

• Quality Assurance in an Open Project
• Social media is changing the shape of scientific debat
• Wikipedia – 25 things you should know
• How Wikipedia’s model can impact on the world of education
• Best Wiki-moments