SCMS - Semantic Content Management Systems for Enterprise Knowledge Management & News Mining » Andreas Blumauer

Survey result: Controlled Vocabularies Matter!

Andreas Blumauer — Mon, 04 Jul 2011 09:41:22 +0000

As announced in another blog post a survey was conducted by Semantic Web Company which should find out how controlled vocabularies are perceived and applied by information managers today. Some of the results are covered by a blog post titled “Thesaurus based search engines will become main stream in the near future“, the survey results can be downloaded here.

PoolParty as a provider of standards based tools for

knowledge organizsation,
semantic search,
linked data management,
knowledge modeling and
text mining

is pleased with the results because they reveal that industry is heavily interested in open standards like SKOS or RDF and sees the value of linked data based on W3C´s semantic web stack. Here is a short extract of the survey:

Do you think enterprises and other organizations can significantly benefit from using Linked Data?

The answer is a clear YES. A subsequent question also reveals that all kind of organisation sizes have about the same opinion concerning linked data. Only few people think that linked data is a “niche thing”. In general it can be said, that over 90% of the participants think that most or at least some organisations can benefit from using linked data.

Florian Bauer: I like to view “linked data” as a “single worldwide API”

Andreas Blumauer — Wed, 16 Mar 2011 14:18:15 +0000

Florian Bauer is REEEP’s Operations and IT Director, responsible for the overall operational management of the organisation, the product management of reegle (the search engine for renewable energy and energy efficiency) and the management of the IT landscape of REEEP. PoolParty Team had the chance to talk with Florian about reegle – information gateway on clean energy. Could you please give us a brief overview over reegle – what are the targets you are pursuing with this platform? The main aim of the reegle information gateway (http://www.reegle.info) is to provide a one-stop gateway to comprehensive, high-quality and up-to-date information on clean energy. By making this information accessible to stakeholders in the field around the world, and by presenting it in a user-friendly and intuitive format, reegle directly helps to facilitate the transition to low-carbon energy. The website provides information on renewable energy, energy efficiency and climate change and their various sub-sectors at a global level, and some reegle services actually combine raw data sets from several different sources, put these datasets into context and thus provide enriched information. reegle is an offshoot of the Renewable Energy & Energy Efficiency Partnership (REEEP), a non-profit, specialist change agent aiming to catalyze the market for renewable energy and energy efficiency, with a primary focus on emerging markets and developing countries. The new reegle data portal (data.reegle.info), launched in 2011, has established reegle as a publisher and consumer of Linked Open Data in the energy sector. It provides key clean energy datasets free for re-use using Linked Open Data W3C standards. reegle consists of two components: one is the semantic search engine (http://www.reegle.info/), the other is the linked data portal (http://data.reegle.info/) – What are your target groups, and which typical problems of the clean energy domain can you solve with these services? For reegle.info, our target groups are primarily project developers, financiers and government policy-makers. These users can access high-quality information on clean energy-related issues with the set of tools we provide: a special web search, a catalogue of more than 1700 key stakeholders, a map view for geographical browsing, a clean energy glossary, and an energy country profiles function. The energy country profiles are typical of what we’re trying to achieve. Here, we take information from many different providers and combine it all to present one comprehensive information dossier on renewable energy and energy efficiency in that particular country. This means that in one location you have the country’s most important energy-related information ranging from key statistics, and current regulations to key players in the energy field in both public and private sectors. For our data portal, the target group is a more technical one: primarily IT developers and open data specialists who want to create new mash-ups and integrate data from reegle into other websites. One of the first using these reegle data sets is the OpenEI.org website, another key portal in the energy field. Open data is not the same as linked open data. Why did you choose to build your services around W3C´s linked data paradigm and/or standards like RDF? Tim Berners-Lee once mentioned that he likes to compare the progressive ways of offering data with the “stars system” used to rate hotels. You get: * for making data public (in any format) ** for machine-readable formats (structured data) *** if the data is offered in a non-proprietary format **** if you use URIs to identify things, so people can point to your datasets ***** for linking to other people’s data to provide context So, as you can imagine, our goal is for reegle to be firmly in the 5-star category, and to establish reegle as an avant-garde tool in energy data. I also like to view “linked data” as a “single worldwide API”. If the old web was like a huge book, the new semantic web is like a huge database, and SPARQL is the way to ask for information – by sending a query through the SPARQL Endpoint. RDF is the language that offers all possibilities to describe a given dataset with all of the necessary information, including any links to other datasets. Therefore RDF data and SPARQL endpoints provide a powerful tool to find and filter datasets and are crucial, base parts of the semantic web’s architectural layers. On reegle the SPARQL endpoint and the description of the structure of our RDF files is online on our clean energy open data portal. You also decided to build a SKOS based domain thesaurus for clean energy which now plays an important role to improve the search experience at reegle. Which experiences have you gained so far from this effort? Which obstacles did you have to overcome? The SKOS-based renewable energy thesaurus can be seen as the “heart” of reegle as it provides the basis for a lot of related services in reegle, including the refinement suggestions for search results, the auto-completion options and the glossary links between defined terms and their synonyms and related terms. We decided to use SKOS because we think it is the best language for building a formal and controlled vocabulary for thesauri in a semantic web context, without adding too much complexity. Although it is a simple language, you really still need IT experts to use it to build a thesaurus – domain experts with additional IT skills (hard to find!). So in our case, we decided to use a scalable and easy-to-use thesaurus server called “PoolParty”. Using this system drastically reduced the complexity, and allowed us to concentrate on the actual building of the thesaurus with our domain experts, and to spend less time on transferring the knowledge into data sets. What are your future plans with reegle? Currently we’re working on restructuring the site to better highlight our new added-value services such as the clean energy country profiles. We are also planning to further develop our thesaurus to include climate-compatible development terms and we’ll soon release a wordpress plug-in to insert this thesaurus into clean energy blogs. One of the most exciting projects we are actually working on is the development of “dossier pages”, where we will provide relevant information to several topics mashed up on one page using semantic web technologies. This is part of the EU funded SCMS (“semantic content management system”) project.

Les Kneebone: “Semantic web technologies are one solution to linking education data in Australia”

Andreas Blumauer — Mon, 15 Nov 2010 07:38:36 +0000

Les Kneebone is Project Manager at Education Services Australia Ltd.
Among other projects he is responsible for Schools Online Thesaurus (ScOT).

PoolParty Team asked Les a couple of questions about thesaurus management, linked data and the semantic web. Here is a short summary of this interview:

Why did you choose thesauri to organize your information? What kind of problems are you able to solve with this approach?

A thesaurus approach was chosen rather than a subject headings approach because we assumed (and continue to assume) that post-coordinate indexing will drive vocabulary-assisted discovery.

Which role does SKOS and/or Linked Data play in order to achieve your goals?

ScOT concepts are now published as URIs. This approach solves the problem of different ScOT versions in disparate systems.

What are the most important values you generate for your stakeholders? What kind of applications can be built or have been built on top of your thesauri?

The Achievement Standards Network (ASN) provides a model for profiling curriculum statements and linking those statements to education resources using various rdf vocabularies. By profiling curriculum statements to learning resources, more precise matching is achieved.

What are the most important arguments to use Semantic Web standards and linked data, especially in education?

The Australian education sector is characterized by many disparate systems in different education jurisdictions. Semantic web technologies are one solution to linking education data in Australia.

Why did you choose PoolParty to manage your thesauri?

We had already identified SKOS as an important standard for ScOT so it was natural to select PoolParty as a our new thesaurus management tool.

What are your future plans and next steps? How do you manage to get your thesauri used, how are you going to build an “eco-system” around your work? (Do you plan to publish ScOT on the LOD cloud? Under which licenses?)

Our vocabularies are currently for non-commercial use and we don’t anticipate any change to the license at this stage. The ScOT license requires attribution, permits derivatives that must be shared, and is for non-commercial use.

Read the full interview here.

Stella Dextre Clarke & Alan Gilchrist about the “Future of Knowledge Organization on the Web”

Andreas Blumauer — Mon, 21 Jun 2010 06:00:10 +0000

Semantic Web Company (SWC) had the pleasure and the opportunity to talk with two internationally recognised experts in the fields of information management and knowledge organization: Alan Gilchrist and Stella Dextre Clarke. SWC asked some questions about the “Future of Knowledge Organization on the Web & Linked Data” on the occasion of an event of the same name organised by ISKO UK which will take place on September 14, 2010 in London. 1. Alan, you are one of the leading experts in the field of thesaurus construction. Organising knowledge in a (worldwide) Semantic Web is a rather young discipline compared to your domain. What do you think can the Semantic Web community learn from “traditional” thesaurus management and vice versa? You put inverted commas round the word traditional, but it might be more appropriate to put them round the word thesaurus! So long as words are used in information retrieval and in information sharing, different forms of structured vocabularies will be required, and many of the fundamental principles of thesaurus construction are still valid for their construction. Of course, the “traditional” thesaurus has mutated since the days when it was used only for controlled indexing and retrieval; and now, with the many enrichments possible it can be viewed as an ontology (in one of the definitions of this word). What remains a difficulty is to create a generalisable typology of associative relationships, though this is, of course, possible in relatively closed systems. In short, structured vocabularies with broadly thesaurus formats will be a necessary component in the web stack. 2. Stella, as a consultant you are specialized in the design and implementation of knowledge structures for information retrieval applications. In the last few months we have seen that SKOS can serve as a significant building block to link “traditional” thesaurus management to knowledge structures from the semantic web. Can you see that this development is market-driven, is there a significant growth of demand for solutions built around SKOS? This question sounds surprisingly sceptical about the growth of SKOS. I guess the dizzying speed of phenomena like Facebook and Twitter has fuelled expectations of tools springing up overnight like mushrooms, fully formed and ready to eat. But actually it takes time, not just for the tools to be fashioned, but for the potential market to develop an understanding of what they can do and what will happen next when they are used. Applications for SKOS are springing up all the time, as fast as people can grow the skills and vision to deploy them. At the moment the market, or shall we say the power-base, seems to be with the academic sector and allied not-for-profit organisations. This will spread progressively through the public to the private sector, as enterprises find ways of adapting their business models. The main hurdles to overcome could be intellectual property rights and the need for compilers of databases to keep earning their living. 3. Alan, constructing thesauri for the semantic web also means that one has to make the “open world assumption”. In which sense does this change the way to manage thesauri, keep them growing and assure quality? Can you see new, upcoming methodologies to do that? Everything changes with the “open world assumption”! Following on from my answer to the previous question, it seems clear that one manifestation of the thesaurus will be found in those systems that support interoperability, such as federated searching or metadata registries. Even with simple thesaurus management software, it is possible to construct a “master vocabulary” or “word bank” to support different applications within an enterprise; thereby promoting interoperability. More sophisticated software is already available (though not very widely); more will be needed and, doubtless, will be created. A more formal answer to both questions will be found in a new standard – ISO 25964, currently being prepared on the basis of BS 8723. The two fundamental features of these two standards are (1) the thesaurus as a theoretical and practical basis for the construction of structured vocabularies for information retieval and (2) the growing and vital need for interoperability between systems and the intelligent mapping of the vocabularies used by those systems. 4. Stella, just recently at ESWC 2010, Sean Bechhofer was asked during his keynote why there are so few SKOS tools on the market. What do you think are the reasons for this? Are there still shortcomings of the SKOS specification compared to other existing thesaurus standards? (see also: http://www.eswc2010.org/program-menu/keynote-speakers/155-sean-bechhofer & http://www.slideshare.net/seanb/skos-past-present-and-future ) Regarding the speed of development, see my reply above. As to shortcomings, did you note in one of Bechhofer’s slides: “Standardisation is necessarily a compromise: Everyone equally unhappy = success!” The SKOS development team took a conscious decision to keep the schema sufficiently simple that it could be applicable to as many different types of KOS as possible. On the downside, this means SKOS is unsatisfactory for conveying sophisticated features of some thesauri and classification schemes. But by keeping the entry barrier low, more widespread use has been encouraged. By way of illustration, compare SKOS with the data model and XML schema of BS 8723. This schema is comparatively specialized, with the aim of enabling exchange of any thesaurus carrying any or all of the features recommended in the standard. And incidentally, this data model and schema will have some further capabilities added when published in the forthcoming standard ISO 25964. SKOS does not provide for a number of features in these standards (such as compound equivalence). But the schemas in BS 8723 and ISO 25964 are designed for thesaurus developers to share their work, rather than for easy publication on the Web, and will never have so many users or associated tools as SKOS. So I believe that SKOS has done well to accept compromises that encourage generalisation although they might not suit some specialists. That said, I do regret one of its weaknesses in the context of mapping. Compound equivalence mappings (that is to say, where Concept A in one vocabulary maps to a combination of Concepts B and C in another) are very commonly needed when extending a search across multiple databases, and the SKOS mapping properties do not currently allow for them. Perhaps there will be some provision in future? 5. Stella, Alan, in September ISKO UK will organise an event on “The Future of Knowledge Organisation on the Web”. “Linked Data” seems to be a promising approach to organise knowledge in large scale environments. Could you imagine that SKOS as a small subset of semantic web specifications will play a central role in this environment since it is quite intuitively comprehensible by virtually any knowledge worker or do you rather think SKOS is too simple (or too complex)? (see also: http://poolparty.punkt.at/using-skos-as-an-interface-to-the-linked-data-cloud ) Stella: Of course SKOS will have a central role (whether or not every knowledge worker finds it as intuitive as you suppose). “Linked Data” will find even wider applicability. ISKO-UK (the organiser of the meeting in London on 14 September) has a mission not just to spread the word about both these technologies, but to build bridges between the several communities who must share their expertise and data to build more exciting applications. We’re expecting an audience of over 100 at this low-cost event. Alan: Yes, of course, just as all the tools in the web stack will be necessary if semantic web technologies are to be effective. But it is obvious that we are dealing with complexities of a higher order than ever before. Any structured vocabulary is an “artificial language” which, while acknowledging many aspects of theoretical linguistics is forced to be pragmatic in its construction. Consequently, it would not be surprising if SKOS is seen to be “catching up”, and this became apparent in the work of BS 8723 when thesaurus models using UML were being constructed. There remains much work to be done on all fronts. Stella Dextre Clarke is an independent consultant specializing in the design and implementation of thesauri and other knowledge organization structures. She currently leads ISO NP 25964, the project to update and revise the international standards for thesauri. Previously she was the Convenor of the Working Group which developed BS 8723. In 2006 she won the Tony Kent Strix Award for outstanding achievement in information retrieval, in recognition for her development work on IPSV (Integrated Public Sector Vocabulary), as well as on the vocabulary standards. She is a Fellow of the Chartered Institute of Library and Information Professionals. Alan Gilchrist has been a consultant for many years in the fields of information management and information architecture, specialising in the vocabulary aspects of information retrieval. He is co-author, with Jean Aitchison and David Bawden of Thesaurus Construction and Use, now in its fourth edition. In 1979 he founded and edited the Journal of Information Science, and is now Editor Emeritus. He has an Honorary Degree (D. Litt.) from the University of Brighton and is an Honorary Fellow of the Chartered Institute of Librarians and Information Professionals.

Kingsley Idehen: “By declaring its context, Linked Data can be made more easily reusable by others”

Andreas Blumauer — Wed, 16 Jun 2010 13:23:18 +0000

Semantic Web Company talked with Kingsley Idehen who is CEO of OpenLink Software and probably one of the most profound experts on data integration issues about “Linked Data”.

The interview covers questions like:

How can Linked Data help to make companies more productive?
Do you think that the Linked Data Initiative can build upon a stable architecture or will it face more and more problems the bigger the “cloud” will grow?
What´s the ultimate argument for an Enterprise Architect to use languages like SPARQL at least in addition to SQL?
How will a “Real Time Semantic Web” change the whole game?
How will the “Semantic Web” be called in 10 years? Will there still be a “Semantic Web”?

Read the full version of the interview here.

Publishing Linked Data for the first time (PoolParty 2.7 release notes)

Andreas Blumauer — Mon, 14 Jun 2010 14:13:23 +0000

I still remember when I was publishing HTML for the first time in my life: It took place in 1996 and I used Microsoft Frontpage. It was exciting because then “I was on the Internet”. Yesterday, around 15 years later something similar happened: I published Linked Data for the first time actively! Eureka! Sure, by using Semantic MediaWiki or Wordpress’s SIOC plugin “I was already on the Semantic Web” – but a lot of data which is produced by such tools is not Linked Data but simple RDF. A closer look at all the datasets in the LOD cloud also reveals that none of them can be edited with an ease, except upcoming DBpedia Live which offers “real-time semantic web”. Conclusio: So far most of the linked data in the LOD cloud was generated by DB2RDF mapping tools like D2R which can only be handled by semantic web experts and technicians. Don´t get me wrong – this is a very important basic layer for the LOD world. All automatically generated datasets like DBpedia are kind of “highways” on the linked data map. Now it´s time to pave the side streets. Just imagine, a teacher would like to publish his knowledge about Italian painters in a way it can be re-used as linked data. Should we tell him to “open an editor, to start typing RDF triples and to upload the file via FTP”? When we started to design PoolParty in 2007 we had people in our minds who would like to contribute actively to producing data for the semantic web. People working for organizations with special domain knowledge are not only able to connect the dots from the linked data highways but also know how to customize such data for their own applications. PoolParty 2.7 offers the following features and functionalities for such tasks:

Linked Data editing: users generate linked data to describe their resources (concepts) on top of SKOS
Linked data lookup: mapping between own thesauri and additional facts from the semantic web The following resources can be used at the moment: DBpedia, Umbel, Yago, DMOZ, LCSH, Geonames & Wordnet; this service is highly configurable – also internal linked data sources can be mapped and used to enrich local thesauri; the lookup service makes use of the very fast TuQS server
Linked data publishing: based on linked data patterns any resource can be published as linked data, ready to re-use for any linked data application; example: http://open.poolparty.punkt.at/Wine/13 which can also be viewed by linked data browsers like Zitgist’s DataViewer
SPARQL endpoint: another way how PoolParty’s RDF data can be accessed by semantic web developers

In addition to these features PoolParty 2.7 comes with some other new features:

Translation support: works for nearly any language and domain with high accuracy – thanks to Google Translate
Online Documentation: PoolParty’s end-user manual is open for the public, easy to access and searchable; since PoolParty 2.7 it is available not only as PDF document but also as browsable Wiki
Flexible Reporting Tool: As we have already blogged before, PoolParty’s new reporting tool is flexible enough to manage to export formats like, for example, Google Synonyms; also “traditional” thesaurus reports like hierarchical reports are available
iPhone front-end: If you have to do research using your thesauri while you are somewhere outside of the office, this could be a possible solution for you – see this screenshot!

If you also want to publish some linked data (for the first time in your life ) register to get a PoolParty demo account and go for it! It´s really easy.

Using SKOS as an interface to the Linked Data Cloud

Andreas Blumauer — Tue, 01 Jun 2010 17:32:37 +0000

PoolParty 2.7 offers new and comfortable ways to enrich any SKOS thesaurus with additional facts from the semantic web (see: LOD cloud). This functionality (which was extended significantly with version 2.7 in June 2010) supports any thesaurus manager to generate much richer knowledge models (ontologies) around specific domains than ever before (without facing high extra costs due to additional research). There are at least three arguments why one should consider building such “extended thesauri”:

Use even more metadata to describe your resources and improve navigation and semantic search functionalities significantly
Publish (at least) parts of your metadata / knowledge models as linked (open) data to stimulate innovative services around your contents on top of network effects
Use linked data for data integration and semantic mashups; combine your own contents with contents from the web to improve your business intelligence

A short example: Just imagine you are working on a knowledge model in the area of “Skiing in Austria”. You have stated that one of Tyrols´s (most famous) skiing areas is “Kitzbühel“. By looking up at geonames.org you get extra metadata, e.g. alternate labels like “Kitzbichl” or longitude and latitude to display the concept on a map. In a next step you add famous Austrian skiers like “Hermann Maier” and “Franz Klammer“. From DBpedia you retrieve additional category information like Maier is a “Person born in 1972“, thumbnail pictures and also some links to other linked data sources, e.g. to the New York Times. Here we can learn that the NYT has mentioned Hermann Maier in 14 articles already. Finally we can add “Toni Sailer” as a third skier and we will find out by harvesting linked data that he was born in Tyrol and therefore we can add a new relation in our thesaurus between him and Tyrol.

We have learned: Linked Data can help us to build expressive knowledge models by using SKOS as an “interface” to the Linked Data Cloud.

SKOS thesauri can not only serve as a backbone for rich metadata structures to improve search applications but also as a new linked data source to be published and to be linked with other semantic data. PoolParty 2.7 follows many suggestions from “Linked Data Patterns” (edited by Leigh Dodds and Ian Davis) how linked data should be published. For instance, there are various ways with PoolParty 2.7 to identify resources, e.g. via “Patterned URIs” or via “Literal Keys” (see, for example, http://vocabulary.semantic-web.at/SemanticWebThesaurus/controlledvocabulary). PoolParty uses TuQS as very fast linked data lookup service and can harvest data from virtually any linked (open) data source which provides a SPARQL-endpoint, e.g. DBpedia, Geonames, Wordnet, UMBEL or PoolParty sources themselves.

Don´t forget: SKOS stands for Simple Knowledge Organization System, thus PoolParty was designed as an easy-to-use Linked Data and Thesaurus Editor and Publishing System.

PoolParty releases Wordpress plugin to build glossaries for blogs

Andreas Blumauer — Fri, 30 Apr 2010 14:00:04 +0000

PoolParty Team has released a Worpress plugin which imports SKOS thesauri and automatically generates a glossary for any given blog. This works as follows:

the blog admin has to install the “PoolParty Wordpress glossary plugin”
the blog author has to generate a thesaurus which describes, and structures the blog content
the thesaurus has to be exported in SKOS format
the file has to be imported into the blog
a glossary will be generated automatically
all blog posts and pages which contain thesaurus concepts will be linked automatically to the glossary

Please, take a look to the glossary of this blog as an example for this software.

Download Wordpress-Plugin: PoolParty Thesaurus for Wordpress 1.0 (or go to Wordpress plugin dircetory)

Thanks to Benjamin Nowack: The thesaurus is imported and into the system and is queried via ARC2.
Thanks to rduffy. His ‘Glossary’ Plugin inpired us, and we were able to develop this plugin on top of his ideas.

Works with PHP 5, MySQL 5 und ARC2

Installation

Download the plugin zip file
Upload the plugin contents into your WordPress installation’s plugin directory.
The plugin’s .php files, readme.txt and folders should be installed in the ‘wp-content/plugins/pp-thesaurus/’ directory.
Move the ‘pp-thesaurus-template.php’ file into your active theme direcory
Download ARC2.
Upload the ARC files and folders into ‘/wp-content/plugins/pp-thesaurus/arc/’ directory.
From the Plugin Management page in Wordpress, activate the ‘PoolParty Thesaurus’ plugin.
Create a main PoolParty Thesaurus page (example “Thesaurus”) with or without body content.
Create a child page (example “Item”) of the main PoolParty Thesaurus page with or without body content and take the ‘PoolParty Thesaurus’ template.
Go to ‘Settings’ -> ‘PoolParty Thesaurus’ in Wordpress, enter the main Thesaurus page’s id# and import a SKOS/RDF file

Frequently Asked Questions

Does my main PoolParty Thesaurus page need to be titled “Thesaurus”?
No. It can be called however you like. Just make sure to enter the page’s id into the plugin’s settings dashboard.

Does my child page need to be titled “Item”?
No. It can be called however you like.

How do I add a thesaurus item?
Therefore you need a SKOS thesaurus management tool like PoolParty. The glossary is generated automatically from the imported thesaurus.

How can I update the glossary?
Simply load the updated thesaurus again (admin area – settings -> PoolParty Thesaurus). The old thesaurus will be overwritten. New or updated concepts will be recognized immediately by the link generator.

PoolParty 2.6 available now!

Andreas Blumauer — Tue, 23 Feb 2010 13:06:53 +0000

PoolParty 2.6 was released today with useful new features:

Special import functionalities: Each concept can be populated with multi-lingual lists of sub-concepts via CSV import.
User Management: Each PoolParty user belongs to one or many usergroups which can own and edit one ore many thesauri. Projects published into the “public” group can be viewed via wiki frontend without login.
New skin: Beside “White Russian” and “Tequila Sunrise” and new skin called “Bluebird” is available now (all skins are named after cocktails)