• Information on generation of snapshots
• Admin Dashboard offering information for projects
• Custom parameters for queries in the SPARQL endpoint.

To get an overview on all changes made in Release 2.8.1 you can read the Release Notes.

Try PoolParty and get a Demo Account or join our next webinar to learn more about PoolParty.

Les Kneebone is Project Manager at Education Services Australia Ltd.
Among other projects he is responsible for Schools Online Thesaurus (ScOT).

PoolParty Team asked Les a couple of questions about thesaurus management, linked data and the semantic web. Here is a short summary of this interview:

Why did you choose thesauri to organize your information? What kind of problems are you able to solve with this approach?

A thesaurus approach was chosen rather than a subject headings approach because we assumed (and continue to assume) that post-coordinate indexing will drive vocabulary-assisted discovery.

Which role does SKOS and/or Linked Data play in order to achieve your goals?

ScOT concepts are now published as URIs. This approach solves the problem of different ScOT versions in disparate systems.

What are the most important values you generate for your stakeholders? What kind of applications can be built or have been built on top of your thesauri?

The Achievement Standards Network (ASN) provides a model for profiling curriculum statements and linking those statements to education resources using various rdf vocabularies. By profiling curriculum statements to learning resources, more precise matching is achieved.

What are the most important arguments to use Semantic Web standards and linked data, especially in education?

The Australian education sector is characterized by many disparate systems in different education jurisdictions. Semantic web technologies are one solution to linking education data in Australia.

Why did you choose PoolParty to manage your thesauri?

We had already identified SKOS as an important standard for ScOT so it was natural to select PoolParty as a our new thesaurus management tool.

What are your future plans and next steps? How do you manage to get your thesauri used, how are you going to build an “eco-system” around your work? (Do you plan to publish ScOT on the LOD cloud? Under which licenses?)

Our vocabularies are currently for non-commercial use and we don’t anticipate any change to the license at this stage. The ScOT license requires attribution, permits derivatives that must be shared, and is for non-commercial use.

Read the full interview here.

• Import and export of subtrees and concept schemes
• Creating subproperties for relations
• Adding notes to concepts (Change/Editorial/History notes)

To get an overview on all changes made in Release 2.8 you can read the Release Notes. The User Guide for the new PoolParty Release is also available.

Try it out and get a Demo Account or join our next webinar to get a deeper insight.

The interview covers questions like:

• How can Linked Data help to make companies more productive?
• Do you think that the Linked Data Initiative can build upon a stable architecture or will it face more and more problems the bigger the “cloud” will grow?
• What´s the ultimate argument for an Enterprise Architect to use languages like SPARQL at least in addition to SQL?
• How will a “Real Time Semantic Web” change the whole game?
• How will the “Semantic Web” be called in 10 years? Will there still be a “Semantic Web”?

Read the full version of the interview here. ]]> http://blog.semantic-web.at/2010/06/16/kingsley-idehen-i-only-think-in-terms-of-a-web-of-linked-data/feed 0 http://blog.semantic-web.at/2010/06/07/lyndon-nixon-with-the-hundreds-of-tv-channels-available-content-selection-becomes-a-significant-challenge-for-users/ http://blog.semantic-web.at/2010/06/07/lyndon-nixon-with-the-hundreds-of-tv-channels-available-content-selection-becomes-a-significant-challenge-for-users/#comments Mon, 07 Jun 2010 10:25:02 +0000 Tassilo Pellegrini http://blog.semantic-web.at/?p=1600 From June 9 – 11, 2010 the EuroITV Conference discusses latest advances and research of media technology, HCI, media studies, and the content creation community. Tassilo Pellegrini talked to Lyndon Nixon, STI International, about the future role of semantic technologies in the television industry and how a Social Semantic Web might influence the traditional television experience.

At this year’s EuroITV conference you will hold a workshop on the EU project NoTube. Can you give us a brief insight what this project is about?

What promises does the Social Semantic Web hold with respect to innovate the television experience? What is the vision?

What economic effects on the value chain do you expect from semantically empowered television? Will there be new revenue opportunities with respect to advertising or Pay TV models?

Despite many attempts experience has shown that television is a rather conservative and innovation-averse medium. What can be done to stimulate the uptake of semantic technologies in the television sector?

Workshop Information

About Lyndon Nixon

Well, in the previous section we mentioned in passing that our technology was based on a synergistic approach, combining syntax, semantics and pragmatics. In this final part of the survey, we’ll explain just how we do this, and why our system yields unparalleled results. In doing so we’ll do our best to abstract away from the underlying mathematics and details of the machine-learning algorithms, and instead present the linguistic principles by which our algorithms work using examples. To wrap things up, we’ll end this review with a snapshot of what Digital Trowel’s Sentiment Analysis looks like in action.

Our technological approach begins with the observation that sentiment is conveyed on three interacting levels of increasing structural complexity. Namely the lexical, phrasal and semantic-event level of structure. We’ll explain.

Lexical sentiment, sometimes referred to as dictionary-based sentiment, is the sentiment attributed to single isolated words. For example:

great, wonderful, terribly, worrisome, helpful, etc…

Though single words clearly carry sentiment, this is the most rudimentary and least reliable sentiment available. To see this consider the following phrases using the above examples:

Great failure
Wonderful fiasco
Terribly surprising comeback
Worrisome transformation for previous skeptics
Helpful in expediting the demise

It should be evident from the above phrases that the initial or “natural” sentiment associated with the isolated words, have all been transformed if not negated. To avoid such “wonderful fiascoes” in deciphering the sentiment, we employ the lexical analysis of sentiment only after the text has undergone syntactic parsing. In simple words syntactic parsing means that sentences are analyzed to determine their grammatical structure and that each word is assigned its corresponding Part Of Speech (POS) tag.

Consider the following example taken from Cisco’s website (where red and green indicate negative and positive sentiment, respectively):

If Cisco does not achieve the desired level of acceptances, the company will withdraw the offer and evaluate alternative ways to expand our activities in the video communications market.

To glean the lexical sentiment, the sentence is first parsed, i.e. grammatically analyzed. For starters, this allows us to determine the subject of the sentence (“Cisco”) as well as any pronominal phrase referring to the subject (“the company”) – both of which have been marked in bold above. Naturally, this is of critical import to us is in determining what company the sentiment is to be associated with. Secondly, once we obtain a phrasal structure of the sentence we are able to determine how a candidate lexical entry interacts with clause-mate entries. In the example above, “desired” is typically associated with positive sentiment, but this sentiment is reversed due to the negation “does not” appearing earlier in the clause. On the other hand in the subsequent clause the verb entries “evaluate” and “expand” maintain and even substantiate their positive sentiment, as there is nothing in the clause to alter their natural interpretation.

Obviously, not all lexical entries are born equal. Entries may vary both in the extent to which they convey a sentiment and their relative intra-clausal effect. For example “excellent” conveys a stronger sentiment than “good”, whereas “great” and “superb” generally indicate the same level of positivity, but “great” is more susceptible to lexical negation (cf. “great mistake” vs. “superb mistake”). Different entries therefore receive different weights, depending on their relative sentimental strength and susceptibility to polarity-transformations. In order to correctly assign weight to these words, DT uses advanced statistical models which are generated using large manually-analyzed text corpora. In addition further factors such as conditional, speculative and contra-factual clause structures are taken into account before the final contribution of specific entries are calculated.

But this is only the first and most rudimentary element of our synergistic approach.  The second more complex element is that associated with the phrasal level of structure. The phrasal level of analysis assigns a sentiment value to full phrases rather than to single words. Consider the following examples:

Cisco Chief Executive John Chambers has said the firm aims to gain market share in a tech recovery.

Boosted by those moves and  …  following last year’s 40 percent decline

Company Struggles in Attempt to Buy Time

In the examples above the lexical level may signal certain entries are positive or negative, but  only a real phrase-level analysis can ascertain the sentiment. It is here that we first allow semantic and pragmatic factors to interact. It is not enough to understand the meaning of each word in isolation, the meaning of the entire phrase must be deciphered, and to so  correctly, context is needed.

Take a look for instance at the third example above. Usually when companies buy something, it’s either a product or another company. Here, however, it is clear that an idiomatic meaning is intended (buying time… stalling).

DT’s SA takes pragmatics to a whole new level. Not only do we use carefully developed word-classes to allow our engine to utilize outside knowledge in interpreting text, but, working with a team of linguists and economists we have developed specialized sets of phrase level interpretive rules, which allow the engine to identify the context of a sentence or phrase.  All of this combined with the simple pragmatic module which is used to identify key companies by resolving anaphora and common nicknames and descriptors and you end up with a context identifier that allows our engine to assign sentiment to even highly complicated, idiomatic or obscure phrases. Believe it or not, allowing our semantic and pragmatic modules to collaborate, our engine is able to pick up on sarcastic, wishy-washy, and even ironic notes in the text.

This brings us to the third level of our Synergistic Sentiment Analysis, which is based on the interpretation of actual events within the text. Transcending both lexical and phrasal levels of interpretation, we have trained our engine to identify key economic events, and together with a team of experienced financial experts, we’ve created a scale of positive and negative weights for these events. Take a look at the following examples:

shares of Cisco Systems (Nasdaq: CSCO) were recently up 47 percent

Cisco expects revenue to grow 1 to 4 percent

Cisco(R) (NASDAQ: CSCO) today announced a revised recommended voluntary cash offer to acquire TANDBERG (OSLO: TAA)

All the above are real examples of events captured by our SA engine and marked as positive. We currently have our engine trained to extract and evaluate dozens of types of events including purchases, stock offerings, workforce changes, legal events, product launches or recalling, hiring and firing of key figures, new facilities, bankruptcy, etc… etc…

The event-level of our SA assigns the highest weights since it combines and epitomizes all of our techniques. Using syntactic, semantic and pragmatic analyses to determine the contribution of the event to the sentiment. In fact, we believe that by identifying and analyzing the key events in the text we are emulating just what an expert would do when attempting to estimate the sentiment associated with a given text excerpt.

Starting from the lexical level, which allows us to pick up on subtle tones in the text , building up to phrases which indicate attitude, and embedding these all within a semantic-pragmatic event extractor and economic-analyzer, we believe we are truly able to capture the sentiment of text very much like a human would, with incredible reliability and consistency. We may not have yet passed the Turintg Test, but we’re surely on the way to improve the ability of machines to “understand” the natural language that humans use!

Well, for now that’s all we can show, without divulging too much

1. the blog admin has to install the “PoolParty Wordpress glossary plugin”
2. the blog author has to generate a thesaurus which describes, and structures the blog content
3. the thesaurus has to be exported in SKOS format
4. the file has to be imported into the blog
5. a glossary will be generated automatically
6. all blog posts and pages which contain thesaurus concepts will be linked automatically to the glossary

Please, take a look to the glossary of this blog as an example for this software.

Download Wordpress-Plugin: PoolParty Thesaurus for Wordpress 1.0 (or go to Wordpress plugin dircetory)

Thanks to Benjamin Nowack: The thesaurus is imported and into the system and is queried via ARC2.
Thanks to rduffy. His ‘Glossary’ Plugin inpired us, and we were able to develop this plugin on top of his ideas.

Works with PHP 5, MySQL 5 und ARC2

Installation

1. Download the plugin zip file
2. Upload the plugin contents into your WordPress installation’s plugin directory.
3. The plugin’s .php files, readme.txt and folders should be installed in the ‘wp-content/plugins/pp-thesaurus/’ directory.
4. Move the ‘pp-thesaurus-template.php’ file into your active theme direcory
5. Download ARC2.
6. Upload the ARC files and folders into ‘/wp-content/plugins/pp-thesaurus/arc/’ directory.
7. From the Plugin Management page in Wordpress, activate the ‘PoolParty Thesaurus’ plugin.
8. Create a main PoolParty Thesaurus page (example “Thesaurus”) with or without body content.
9. Create a child page (example “Item”) of the main PoolParty Thesaurus page with or without body content and take the ‘PoolParty Thesaurus’ template.
10. Go to ‘Settings’ -> ‘PoolParty Thesaurus’ in Wordpress, enter the main Thesaurus page’s id# and import a SKOS/RDF file

Frequently Asked Questions

Does my main PoolParty Thesaurus page need to be titled “Thesaurus”?
No. It can be called however you like. Just make sure to enter the page’s id into the plugin’s settings dashboard.

Does my child page need to be titled “Item”?
No. It can be called however you like.

How do I add a thesaurus item?
Therefore you need a SKOS thesaurus management tool like PoolParty. The glossary is generated automatically from the imported thesaurus.

How can I update the glossary?
Simply load the updated thesaurus again (admin area – settings -> PoolParty Thesaurus). The old thesaurus will be overwritten. New or updated concepts will be recognized immediately by the link generator.

The Space Between the Lines

Welcome back! Sit down and buckle up for a magical tour of the text mining technology focusing on Sentiment Analysis (SA).

Well, first things first. What is Sentiment Analysis anyway? Rephrasing the Wikipedia definition, Sentiment analysis (sometimes called opinion mining) refers to an area of Natural Language Processing (NLP), which aims to determine the attitude of a writer with respect to some topic. This attitude may be their judgment or evaluation, their emotional state when writing or the intended emotional communication the author wishes to convey.

To keep our discussion as concrete as possible we’ll use real life examples to elucidate the different types of attitudes.  Consider the following example:

This year was a setup year for B&N, and 2010 will see its efforts start to pay off [...] In 2010, B&N will rack up significant sales of Nooks and e-books, as some consumers look for an Amazon alternative.

Obviously this excerpt contains an explicit positive evaluation for Barnes and Noble for 2010, but moreover the tone is upbeat, optimistic, and even excited. A good Sentiment Analysis would pick up on this tone and report a highly positive sentiment for B&N and their e-reader Nook, whereas a negative or at least an apprehensive sentiment should be reported for Amazon.

The next example is even more blatant:

Belated Happy New Year and already what a year it’s turning out to be for eReaders! [...] Time’s a fave around here these days, especially considering its December report naming nook one of the Best Travel Gadgets of 2009 as well as rating the device # 2 among the Top Ten Gadgets of the year. While emphasizing nook’s “classy book-lending feature”, the magazine also cited “the powerful, flexible Android operating system that the whole package runs on.”

The exclamation mark, the rhythm, the tone, the profuse use of superlatives and positive adjectives all indicate an extremely positive sentiment for the nook product. It is clear that the author has a favorable opinion of the product, and moreover that he is quite eager to share his enthusiasm with the readers.

Obviously these are not the only attitudes that can be found on the web. Other attitudes may include anticipation, sarcasm, doubt, apprehension, cynicism and even condemnation. It’s our nature to focus on the good, so we’ll spare you examples of the negative attitudes (well, I guess it’s also that we prefer to avoid any unnecessary lawsuits ) but the basic idea of what is meant by an underlying attitude should be clear by now.

It’s important to keep in mind that Sentiment Analysis is not severed from the basic meaning of the sentence. Rather, SA picks up on the basic meaning and further capitalizes on the cadence, the tone, the choice of words, and even the absence thereof, to build a complete picture of the message being conveyed. Note that we’ve implicitly drawn a line between some sort of “basic meaning” of a sentence, and the “ultimate intention” of the  message to be conveyed. Let’s try and be a bit more precise and explicit about this distinction.

Formal linguistic theory usually recognizes 3 levels of abstraction for natural language comprehension: Syntax, Semantics and Pragmatics (we are excluding phonology, phonetics and morphology which are irrelevant here). Simply stated, Syntax is the study of the grammatical structure of sentences, Semantics deals with how words are interpreted and how their interpretation is combined to yield the meaning of the sentence, and Pragmatics is the study of how extra-linguistic, real-world knowledge, so to speak, interacts with the basic meaning of sentences to yield the ultimate message conveyed.

So for example, syntactic theories may attempt to explain why the English sentence “I gave that to you” is fine whereas, “You gave that to I” is ungrammatical. Semantic theories may attempt to explain what the meaning of a word such as “tall” is, and how this meaning can be reconciled with seemingly problematic examples such as “I am tall” vs. “The midget is only 4 feet tall”. Pragmatics, goes one step further and attempts to explain how our knowledge of the world, circumstances, etc. play with and alter the meaning of the conveyed message. So for example, although strictly speaking the sentence: “I have 3 children”, does not formally preclude the possibility that I have more than 3, say 5 children, it would generally be considered wrong, or at least odd, for someone who indeed has 5 children to utter the original sentence: “I have 3 children”. To see how this judgment may change with circumstances, imagine Mr. Jones is being interviewed by the IRS, when he is notified by the interviewer that tax benefits are available to anyone with 3 or more children. Under these circumstance, we would probably no longer consider it odd for Mr. Jones to say “I have 3 children”, even if in fact he had 10 children.

So where does Sentiment Analysis fit in this 3-headed theoretical framework? If you’re guessing the answer lies somewhere between semantics and pragmatics, perhaps with a bit of a syntactic-twist, you’re following this introduction just fine. (If, on the other hand, you thought it was limited to the syntax, you may want to go brew yourself a fresh cup of coffee before you reread the last few paragraphs ).

Mirroring the theoretical image portrayed above, Natural Language Processing algorithms consist of syntactic algorithms (most notably Part Of Speech (POS) parsers and taggers), semantic algorithms (e.g. semantic rulebooks and relation extraction algorithms) and finally pragmatic algorithms (including for example, contextual disambiguating algorithms, and world knowledge look-up algorithms, used in automated translators for instance). At Digital Trowel we’ve honed our Sentiment Analysis algorithms to combine the strengths of these 3 disciplines to produce the most reliable and comprehensive understanding of the message being conveyed, reading not only the text itself, but also between the lines, so to speak.

The mathematical implementation of these algorithms is beyond the scope of this introduction, but this by no means should prevent us from taking advantage of the knowledge we’ve gained thus far to see how Sentiment Analysis techniques may harness the power of the different types of linguistic algorithms in an attempt to achieve their goal. In fact the lion’s share of the third part of this survey aims to do just that. For now, suffice it to say that one of the main reasons we at DT believe that our technology is superior has to do with our synergistic approach of integrating syntactic, semantic and pragmatic algorithms. This is why we call it Synergistic Sentiment Analysis (SSA). BTW, for those of you wondering, synergy is the term used to describe a situation where different entities cooperate advantageously for a final outcome (tx, Wikipedia!). There, you now understand yet another word in the titles above

Before ending this part, let’s focus on the goals of SA, or in other words, what SA is good for. Well, in one sentence, as we already phrased it:

Extracting and discerning the underlying sentiment allows us to transform otherwise inert texts into vibrant business opportunities.

There are many other business opportunities for the SA technology, including some of which we’ve already implemented at DT such as evaluating pharmaceutical forums for client’s sentiment about drugs, as well as sports product satisfaction, but I think this is enough hype for now

The third and final part of this introduction to the field of SA, goes a bit deeper into the SA engine itself, and examines the innovative technology unique to Digital Trowel using real examples… Stay tuned, this is where things get really exciting

Sorrowfully, the young woman could not cross the stream for the water had risen and it would ruin her silk robe. Without hesitating the older monk lifted the woman in his arms and carried her across, placing her gently on the opposite bank. The younger monk was taken by surprise – Buddhist monks were not supposed to touch women, let alone carry them – but decided to keep his silence, and the two proceeded on their journey for long hours without exchanging a word.

Finally at dusk the two monks arrived at their lodging and the younger monk could no longer hold his tongue. He turned to the older monk: “Tell me,” he asked, hardly concealing his reproachful tone, “Why did you carry that woman? We monks are not supposed to touch women at all!”

The older monk smiled amusedly and calmly replied: “I have let go of that woman many hours ago. Are you still carrying her in your mind?”

Now I bet you’re wondering what the heck this Zen story has to do with Natural Language Processing.

Well, it does – just read on

It’s time to tell you a little more about the general extraction process we follow at DT.

You see, CARE doesn’t really work alone. Rather, it is assisted by both an HTML converter and a post processor.

The following diagram will help to illustrate the process:

As you can see CARE is “sandwiched” between the HTML Converter and the Post Processor. This blog entry will focus on the Post Processor, but first for the sake of completeness a few words about the HTML Converter:

The HTML Converter can be thought of as a “pre-processor”. It begins by downloading selected HTML code from the internet, and then performs the following tasks:

• Cleans the HTML code leaving only the English text
• Cleans the text of advertisements and other garbage sections
• Classifies the text according to its content and determines what CARE rulebooks should run on each section of the text

So for example, sections describing a person’s biography will be marked and designated for the PPC (Person Position Company) and Education rulebooks, while sections of text containing addresses will be sent to the Contact Information rulebook.

Next, one or more of CARE’s rulebooks are run on the text. The text tagged by CARE is what’s fed to the Post Processor.

The Post Processor is essentially an extremely potent script that runs on CARE’s tagged output and turns it into sensible and readable information. Along the way it performs several intricate tasks, some of which are technical and some semantic in nature. In a less than perfect analogy, the Post Processor can be thought of the younger monk in our story, still holding on to all the events in their original context of which the older monk (aka CARE) has long since let go

Following is a brief survey of the Post Processor’s most notable tasks:

• Performing idiosyncratic regular expression substitutions and fixes
• Resolving co-references and anaphoric phrases
  • For example the post processor will replace pronouns such as “he” or “she” by the relevant name of the person. Likewise if the phrase “the company” appears in an extracted relation, it will be replaced by the appropriate name of the company
• Matching of entities and merging of relations – this is to ensure that information is not unnecessarily duplicated, and that all data is matched with the correct entities and only with them
• Assertion and Filtering of outputted relations – this is a last validation of the information’s integrity

An example of part of the post processor’s capability is in place. Assume CARE is fed the following input sentence:

“Micha Breakstone earned his masters in math at Hebrew University (2007, cum laude), and then went on to study for his Ph.D. in cognitive sciences at Hebrew University. Subsequently he was invited to study for one year of his Ph.D at MIT”

CARE will parse the information correctly, but its output may at times look like gibberish to anyone who is not an expert engineer. Moreover, pronouns and conjunctions must be interpreted, as can be seen:

It is the Post Processor that comes and saves the day, tidying up, putting things in order, and all in all making sense of the myriad of unresolved anaphora and tags, to obtain a completely readable and simple extraction as follows:

<Education>

<DEGREE>masters</DEGREE>

<DISTINCTION>cum laude</DISTINCTION>

<NAME>Micha Breakstone</NAME>

<SPECIALTY>math</SPECIALTY>

<STATUS>earned</STATUS>

<UNIVERSITY>Hebrew University</UNIVERSITY>

<YEAR>2007</YEAR>

</Education>

−

<Education>

<DEGREE>Ph.D.</DEGREE>

<NAME>Micha Breakstone</NAME>

<SPECIALTY>cognitive sciences</SPECIALTY>

<STATUS>study</STATUS>

<UNIVERSITY>Hebrew University</UNIVERSITY>

</Education>

−

<Education>

<DEGREE>Ph.D.</DEGREE>

<NAME>Micha Breakstone</NAME>

<STATUS>study</STATUS>

<UNIVERSITY>MIT</UNIVERSITY>

</Education>

So, as you can see, next week I’m off to MIT

In the next entries we’ll try and focus in more detail on each of the Post Processor’s tasks, discussing strategies for anaphora resolution, relation merging, etc. But meanwhile, wish me luck, continue to ask questions or comment, and in the spirit of Zen, remember to let go of any unproductive burdens you may be carrying in your mind.