Yaron is Writing

• I was at the “NYC wiki-conference 2009“, held on the NYU campus, over the weekend; my thoughts about the conference are here. The one thing I forgot to mention, on a technical note, was a five-minute demo by Tom Maaswinkel, showing a MediaWiki wiki being edited via the soon-to-be-released Google Wave - it wowed the audience, as Google Wave demos tend to always do.
• Jeroen De Dauw released version 0.2 of Maps and Semantic Maps. These new versions have, among other improvements, support for Yahoo! geocoding, and just better-looking code, which is going to be important in the long run, as other developers get their hands on it and start tinkering with the code.
• I added Maps and Semantic Maps to Referata - Semantic Google Maps will be gone shortly. That means mapping on Referata has a lot more options, and it’s already starting to bear fruit - check out the Google Earth option on Food Finds, for instance. Pretty nice!
• Sergey Chernyshev and I released a new version of Semantic Bundle, which now includes Maps and Semantic Maps, replacing Google Geocoder and SGM. It’s really the beginning of the end for SGM, not counting the 30+ wikis it’s already on…
• While working on the new Semantic Bundle version, I had the thought that SMW is starting to feel like a mature technology; in that it seems like the majority of the features that it will eventually have are already in place. The addition of the Semantic Maps extension had a lot to do with it, I think; this was one of the big chunks that I thought was still missing. There are still things left to be done, of course; I have a list of around 30, though they won’t necessarily be features that I implement. And I’m sure there will be various improvements behind the scenes, to speed up queries and the like. But I really feel like the Semantic MediaWiki system of the future won’t look all that different from what it looks like now, with the interplay of categories, templates, forms, properties, External Data calls, tables, maps, calendars, widgets, etc. (whew!) that you can already find in various SMW-based wikis. Though I could be wrong about this.

I’ve been working with Jeroen De Dauw, a student in the Google Summer of Code, on creating a full-scale mapping interface for Semantic MediaWiki for a few months now; by which I mean that he’s done the actual work, and I’ve been around to answer questions and try to bask in the glory. Anyway, I think mapping is crucial for any generic data project, because so much information that we need on a daily basis is location-based, whether it’s information about businesses, people, events, etc. There’s already an extension that handles all this stuff - Semantic Google Maps - but it’s incomplete, first because it relies on Google Maps, which not everyone can use, second because it doesn’t support the incredible Google Earth, and third because it can’t handle displaying locations on non-geographic surfaces (more on that later). Another extension, Semantic Layers, also exists, which uses the open-source OpenLayers mapping service, but it’s had some problems since the beginning that were never fully resolved,

Anyway, yesterday and the day before, Jeroen released the two extensions that he’s been working on, that are meant to provide the generic solution for all of SMW’s mapping needs: they are the Maps and Semantic Maps extensions. Here’s how the two work together: Maps handles the display of individual points, along with geocoding (determining the coordinates of a specific address); and Semantic Maps handles the display of multiple points on a map, defined via Semantic MediaWiki, as well as providing maps as Semantic Forms form inputs. Both support the same mapping services, currently three: Google Maps, Yahoo! Maps and OpenLayers.

Jeroen has been keeping track of all the progress on his blog, which has a lot of information on all of this stuff, including some great screenshots, including this rather breathtaking one of Google Earth being used as a form input.

There’s still a month left in the Google Summer of Code, and Jeroen and I are excited about the extra cushion of time that provides, because it means that there’s an opportunity to add extra features to the system; like being able to show a clickable list of points near each map, so that maps can work more like this; and being able to use OpenLayers to display locations on non-geographic surfaces, such as images. That second one opens up a lot of possibilities, because it allows for things like annotated anatomical charts (see here for an example, from the Semantic Layers wiki) and displaying points on floorplans (see here for an example from the same wiki). For the latter, the example provided is for a video game, although you could easily imagine the same concept being used for more practical purposes, such as displaying events at a conference, or… showing the locations of enemy combatants in a building (hey, I’m allowed to fantasize a little, right?).

By a stroke of good timing, on Saturday I’ll actually be speaking at the New York City wiki-conference (basically a smaller-scale version of Wikimania), on the subject of all this mapping stuff; and hopefully being able to do a Steve-Jobs-at-Macworld thing, where I demo a recently unveiled technology to the crowd. Here’s a link to the panel I’ll be on: “Mapping in MediaWiki”. It’s free to attend, if anyone’s interested.

Okay, all of the stuff I wrote before happened, but it was this time last year, not this year. I was off by an entire year. It’s still cool, though - maybe more impressive, actually, given how much functionality has been added to Semantic MediaWiki, etc. since last year. Anyway, what’s written below is not timely in the least.

—————————————————–

This is cool. The company 23andMe creates reports for people on their genetic profiles - it doesn’t send anyone their entire DNA chain, but just notifies about the presence of SNPs (”snips”), which, as I understand it, are DNA sequences considered specifically informative. (The company’s also known for being founded by Google co-founder Sergey Brin’s wife, but I digress.) Anyway, in April they ran a contest in which they published the 23andMe data for an anonymous woman, and those who took part had to guess at as many of her attributes as possible. The winner was announced three weeks ago, and it was Mike Cariaso, whom I always enjoy talking to, and who runs the site SNPedia.com (”snipedia”). In his winning entry, he gave details for her race, hair and eye color, proclivity for diseases, and more intangible things like personality and intelligence. In their announcement of the winner, the company didn’t say which of the details were accurate, but if even half of them are, it’s a surprising (to me) level of detail.

In any case, the really neat thing is that Mike used SNPedia as the database to get all this information; and SNPedia is a wiki that runs on Semantic MediaWiki, and Semantic Forms. So I think it’s great proof that SMW can compete with any technology out there at the moment as far as enabling open, collaborative databases.(Oh, and the prize is a free genetic screening, which sounds good if you’re into that sort of thing.)

Lots of Semantic MediaWiki-related developments recently…

• SMW now supports a “category” format, recurring events and location-based queries (e.g., to get all points within 10 miles of some coordinate).
• Semantic Result Formats has the new “outline” format.
• Semantic Forms supports creating properties, template, form and category in one step, and using forms to run queries.
• Data Transfer allows for importing both XML and CSV files.
• Replace Text allows for selecting the namespaces in which replacement will happen.
• SMW and a good number of its extensions are now handled by both Admin Links and Configure.
• The SMW quick reference has been updated, both the PNG and PDF version.

I’m pleased to announce my latest extension, Admin Links, released earlier today; which, depending on how you count it, is around my ninth extension (a number I never would have guessed I would reach). I believe this is my conceptually simplest extension yet: just a page of links that are meant to be helpful for administrators. I think that this helps fix a hole in MediaWiki, though: I wrote before that I thought one of the top weaknesses of MediaWiki compared to competing systems was “lack of guidance from the interface about how administrators should accomplish their tasks”. Other applications have wizards, control panels and the like for helping administrators do their daily tasks, but when you first set up MediaWiki, there’s nothing looking back at you but a blank main page, and lots of pages of documentation elsewhere. Admin Links provides the bare minimum, which is a page (at “Special:AdminLinks”) of links to common administrative tasks (like editing the CSS file, managing users, viewing a list of all the wiki’s pages). In addition, for administrators, it puts a link to this page within their “user links”, which are the links usually at the top of the page of “my talk”, “my preferences”, etc.; that way, an administrator can easily get to it from whatever page they happen to be on. Finally, Admin Links provides an API for letting other extensions add on sections and links to the page, so that Special:AdminLinks can always serve as a control panel for whatever set of extensions are installed. You can see an example of Admin Links at work here, on Discourse DB; though, since you’re not an administrator, you won’t see a link to it at the top. I’ve modified my local versions of the Semantic MediaWiki and Semantic Forms extensions to call the Admin Links API already, so you can see a lot of links geared for those two. I plan to check in the new Admin Links code of SMW and SF at some point soon, as well as to add similar calls to some of my other extensions.

The idea for this extension actually came from my wiki hosting site, Referata, which already has such a page for administrators (though there it’s called “Helpful links” - which will probably be replaced by Admin Links soon). And the idea for that, in turn, came because I realized the sheer volume of pages that people creating a Semantic MediaWiki site need to know about was making it hard for people to get started. So, in a very real sense, Admin Links is a Semantic MediaWiki-inspired extension; though of course it will most likely have usage beyond that. I should also note that it was the head of SMW, Markus Krötzsch, who came up with the insightful idea of implementing it as a general extension with an API, back when I discussed it with him a long while ago.

As the Semantic MediaWiki system becomes more mature and better-known, it’s encountering a new (and somewhat exciting) problem: it’s getting increasingly faced off against other applications when large organizations evaluate it as a possible content-management/systems-integration/etc. solution. These other applications include, most notably, Microsoft SharePoint, but also “enterprise wikis” like Confluence and SocialText. And when these matchups occur they inevitably bring the weaknesses and gaps in MediaWiki and SMW into focus. The weaknesses that I’ve personally heard have been raised in this way are:

1. Lack of good WYSIWYG editing (there is a WYSIWYG-editing extension, FCKeditor, that works fine in most circumstances, and I’m in the minority who doesn’t think WYSIWYG editing for wikis is that necessary in the first place, but it’s been brought up as an issue)
2. Lack of discussion forums
3. Little to no access control, for being able to set who can read and/or edit which pages
4. Lack of guidance from the interface about how administrators should accomplish their tasks
5. A boring appearance - most MediaWiki sites tend to look almost exactly like Wikipedia, which itself doesn’t look that exciting
6. Especially for Semantic MediaWiki (as opposed to MediaWiki itself), a skepticism about committing to a system that would require either training internal staff or keeping around consultants indefinitely

Those are the big ones, as far as I’m aware. It should be noted that issues of actual storage and display of data, which take up almost all of the focus of SMW discussions and development, don’t seem to have come up in evaluations of SMW at all; which I think indicates that SMW is far ahead of its competitors on data-related matters. Which is great news, though it does suggest that maybe our efforts should be re-prioritized to some extent.

I have some thoughts on how to deal with all of these, except for the first one, and they’re all worth having a discussion about (#3, the access-control issue, is probably worth having quite a few discussions about). But what I want to talk about in this post is issue #2, the lack of discussion forums in MediaWiki. I’ve heard it mentioned as a concern for three different large organizations in the last month, which I assume means that it’s a big issue and will stay that way until it’s solved.

I think the first thing that needs to be addressed, when talking about discussion forums, is that at least three different things fall into the realm of “discussion forums”, which may help explain why it’s been so hard to get a definitive solution. Here are what I see as the three things:

1. Discussions about wiki pages - questions and conversations about the layout, content, data etc. of the pages in the wiki
2. Discussions about the wiki’s topics - a place for people to talk, vent and argue about the actual subjects of each wiki page, independent of what the wiki pages happen to contain
3. General discussions - forum-like discussions that may be unrelated to anything specifically in the wiki

The first kind of discussion is what MediaWiki’s “Talk” pages are geared for, and generally I think they work fine for that purpose. You could make the case that this system could use some improvement - there’s no reason why users should be able to edit others’ comments, for instance - but I haven’t seen any major problems with them, and extensions already exist, like Liquid Threads, that make Talk pages more forum-like.

The second kind of discussion is unique to public wikis - wikis that are meant to attract a general readership, where there will be a set of users who want to read the contents and comment on the topics, without modifying the content itself. On Wikipedia such comments are simply not allowed, which I think is the right thing to do for a mass-audience reference. But for more-specific sites, meant to attract people interested in one particular set of topics, allowing general venting and discussion makes sense. The current best way to do this, in my opinion, is to have such comments be handled by an outside system. The OpenCongress wiki handles them in such a way: the wiki page on the Employee Free Choice Act, for instance, links to OpenCongress’ main page on this bill (at least, the House version), which itself has a tab for the comments page. The flow could be a little nicer, but the system provides a clear location for comments. Of course, in the case of OpenCongress, the non-wiki site, with comments pages, already existed before the wiki was set up, so it was obvious which approach to take. In the case of a wiki without an external site attached, there’s no good, easy solution at the moment. I believe such a solution is important; I also believe that it should be implemented in some way outside the wiki - in other words, comments should be entered in HTML not wiki text, and they shouldn’t be editable once they’re entered. I also don’t know if comments pages should use the wiki’s user-registration system - commenting systems on blogs and such in general seem to work fine without registration, and I believe it might be important to maintain a separate “identity” between making changes to the wiki and expressing one’s personal opinions. For all those reasons, I think it’s a bad idea to use Talk pages for that purpose, although it’s tempting. (And there’s also the fact that Talk pages are already used for discussions about the wiki content.) So that leaves - some sort of way for comment pages to be integrated into a wiki. This definitely could use more thought and discussion.

The third kind of discussion is just discussions in general, potentially on any topic, that people who read and edit the wiki would want to have specifically with one another. For a private wiki in an organization, this would just be a forum for employees/members to talk; for a public wiki on a specific topic, it would be a forum devoted to that topic. Here there’s the least-strong argument for integrating the discussion directly into the wiki, since plenty of good forum software already exists, like phpBB”, and a MediaWiki extension would never be able to match their functionality (some people have tried creating forms using Semantic Forms to enable such a thing, but I don’t think that’ll ever work nearly as well as dedicated software). However, it’s definitely worth creating, at the very least, a “best practices” document explaining how MediaWiki and forum software should be used together and link to one another; and possibly how to integrate their user-registration systems, using OpenID or anything else.

So that’s what I think about disucussions in MediaWiki. I may get around to writing about the other ones; let me know in the comments if there are any that you specifically want to hear my thoughts on, and of course feel free to share your own thoughts.

Yesterday I released a new version of the External Data extension that allows it to, in addition to previous functionality, get a table’s worth of data (instead of just single values), and extract data from any wiki page holding values in CSV format. The more I think about it, the more I think these additions make External Data among the most important MediaWiki extensions I’ve released (or co-released,since Michael Dale contributed), or maybe even the most important, beating out Semantic Forms; I guess we’ll see.

I hope to write more about the “new” External Data at some point; for now, if you want to hear more about it and what I think its implications are, I’ll be talking about it tomorrow (Thursday) at 1:30 PM EST in session 5 of the semantic wiki conference call “mini-series”. Anyone is free to join in the call. There’ll also be other talks (including three from people I met at the Semantic MediaWiki users meeting in Boston), that should be quite interesting.

Do you like reading about semantic wikis, but really wish you could hear me talking on the phone about them? Well, you’re in luck, because I’ll be speaking in the 3rd session of the semantic wiki “mini-series” of conference calls, on Thursday. The last two sessions, which happened over the last two months, covered the broader world of semantic wikis; this one focuses specifically on Semantic MediaWiki. Markus Krötzsch, the lead developer of SMW, will talk about the core of the technology, and I’ll talk about “Semantic Forms, Semantic Drilldown, Semantic Result Formats, Semantic Google Maps, Semantic Compound Queries and Data Transfer” (evidently, I get bored easily). There will also be people from the Ontoprise corporation presenting their contributions, and some other presenters. Each presentation will also have a real-time slide show on the web. You can see the presentation time and phone number here (it depends on where you live), plus other details, and a place to RSVP (you don’t need to RSVP to watch/listen, but it’s strongly recommended).

Thanks to Sergey Chernyshev, I got a chance to see Microsoft SharePoint for myself last week; he was nice enough to show me a lot of the interface. Or rather, what he demo’d one of the two applications referred to as “SharePoint” - it’s still a little confusing to me, but I believe what he showed me was Windows SharePoint Services, as opposed to “SharePoint Server”. That’s good, because I believe that’s the application usually referred to as a competitor for Semantic MediaWiki. Anyway, seeing the SharePoint I saw, I could certainly understand how comparisons were being made. Windows SharePoint is all about collaborative data - users can create their own page types, with each page containing a set of fields; once you’ve created a page, there’s an automatic form created to let everyone add or edit data. Finally, there are “views” one can create, which are the equivalent of SMW’s queries - you can filter a set of pages by some set of criteria, and decide how to display the pages; just as with SMW, you can make tables, calendars, etc.

SharePoint is really closer to something like a set of public Excel spreadsheets with macros for entering data, or sites like Dabble DB, than to Semantic MediaWiki; with the caveat that SharePoint allows for uploading external files in a manner similar to MediaWiki. There are a few key differences that I can see. First the strengths of SharePoint:

• SharePoint allows for data permissions. You can set who can read and edit and who can’t for nearly each piece of data. At this point, that’s possible only through hacks with MediaWiki, and not at all with Semantic MediaWiki - basically, if someone can read any page on a wiki, they can pretty much read all of it. Is that a big advantage for SharePoint? I’m sure there are a lot of companies that see it that way.
• SharePoint’s interface is very easy to understand. If you want to create a new type of page, there’s a nice wizard that guides you through it in a few easy steps. Pages are “pages”, views are “views”, and that’s all there is to it - there’s no need to understand templates, properties, parser functions or anything else, and the word “semantic” is blissfully out of sight. Contrast that, sadly, with Semantic MediaWiki, where even if you know how wikis work, you still have to spend, I would guess, at least an hour or two reading documentation before you can do a thing.

And the weaknesses:

• SharePoint has no versioning. It is not a wiki. You can’t tell who made which changes to which data and when, and I believe that once a piece of data has been changed its old value is lost forever, except maybe in database archives. As the number of people who can modify a set of information grows, the value of complete versioning grows as well, until you reach the point when you literally can’t function without a record of every single change that was made. That, I think, is a big part of why permissions are so important in applications like SharePoint: you always have to keep the number of people who can change any piece of data to a manageable size; say a few dozen or a few hundred at the very most. Of course, companies can manage this way (they’ve been doing it for hundreds of years), but it’s not ideal.
• In SharePoint, you can’t link data. Every field in a page is a standalone field. If you have a page representing a project, and there’s a field representing the project manager, and that field reads “Bob Hoover”, it’s just a string of letters. It won’t link to a page representing Bob, and there will be no way to connect that information about Bob to anything else we know about him. Yes, you can create a view to find out all the projects that are managed by Bob Hoover, but you can’t go to a page about him and see which projects he manages, plus which other projects he’s a part of, plus his phone number, plus which days he’s willing to carpool. In Semantic MediaWiki, that’s all easy to do.

Note that I’m just comparing the interfaces here - there are obviously huge differences in price, support, etc. etc., but I wanted to give my sense about the applications themselves.

I got back a few weeks ago from the Web 3.0 conference in Santa Clara, California. I found the conference for the most part disappointing, and I think that was mostly caused, indirectly, by the conference’s name. It seems to have become conventional wisdom, over the last year or so, that “Web 3.0″ is a synonym of “Semantic Web”, and that’s fine with me - I think it makes a lot of sense. But now, what does “Semantic Web” mean? As I somewhat knew before, to a lot of people it means two things: the transfer of data among sites through APIs and protocols like RDF (the stuff I do); and the use of artificial intelligence and natural-language processing to try to understand the meaning behind text on the web. In this view of things, the term “linked data” covers the first of those meanings; I don’t know if there’s a term for just the second. In any case, at the Web 3.0 conference the two worlds coexisted, with the first one heavily dominating; for example, the two keynote speeches were given by Peer39 and Powerset, which are, respectively, an ad network and a search engine that do natural-language processing to get better results. I’m sure there’s a lot of usefulness in that kind of text processing, but it’s not relevant to anything I do, and it doesn’t interest me. So I unwittingly signed up for a conference mostly about the many applications of natural-language processing.

There were a few interesting parts: I had some nice conversations, and there were two strange coincidences: I saw a panel presentation by a guy I hadn’t seen since we were kids growing up in Amherst, Massachusetts; and another one by someone whose company was among the first customers of Referata. I got to talk to the first but unfortunately not the second. I also talked a little to the people from Freebase, who, it turns out, had never heard of me or of the world that’s sprung up around Semantic MediaWiki, but then again they’d have no reason to; $40 million in funding (or however much it was) can really help to focus your attention. I also spoke, at a panel on marketing, where I tried to stress the importance of selling to businesses and other organizations, using the tools of Web 3.0 as a solution for so-called “enterprise application integration”. I didn’t get the sense that there was a lot of interest in what I was saying, for whatever reason (maybe because most people were there for the text stuff). I sat next to Thomas Tague of OpenCalais, and I liked what he had to say. Two of his statements stuck with me: that all this semantic technology was probably not going to build a better Google, and more generally, that semantic-web startups seem to resemble 7-year-olds playing soccer: everyone just chases after whoever has the ball (in this case, Google, or maybe Powerset).

The trip itself wasn’t totally a waste: I stayed in San Francisco, with my brother and his family, including the few days before and after the conference. I had a very nice time catching up, and helping out with the kids. I also met up with a few friends of mine, including Nick Grandy, the guy I started Discourse DB with two years ago, who now has his own startup.
The experience may have soured me on business conferences; I guess the Linked Data Planet conference, organized by the same people (Jupiter Media), might be a better fit for me, since it’s more focused on the true-data side of things, as you could guess from the name. I might also end up at the Semantic Technology Conference in San Jose in June. That’s the big one, and it also seems more data-centric. At the last “SemTech” there were at least three presentations that mentioned Semantic MediaWiki: two that mentioned Semantic Forms specifically, one from the Halo developers, and another one that may have, since it included an SMW user on the panel. So my presence there this year would either be very appropriate or redundant.

I’m pleased to state that Verwaltungskoooperation, a semantic wiki that uses several of my extensions, won a jury award at the 2008 Multimedia and e-Business State Awards in Austria. It’s a little hard to understand everything, because it’s all in German, but my basic understanding the site is a collaborative directory of people, organizations and projects that are involved in non-profit and social services around Austria and Germany. The site uses my extensions Semantic Forms, Semantic Drilldown, Semantic Google Maps and Data Transfer. It’s run by the Zentrum für Verwaltungsforschung (Center for Public Adminstration), and specifically Bernhard Krabina, with whom I’ve had a number of conversations. I think this is the first time my extensions have gotten any sort of public recognition, so it’s nice to see; and of course, it’s great news for them.

Swirrl has been launched - it’s a semantic wiki hosting site, which makes it, in my opinion, the world’s second semantic wiki host, after Referata (you could certainly make the case for Wikia being the first, since they had Semantic MediaWiki on their site about six months before Referata was released, but I think their support for it was (and is) primitive enough that it doesn’t really count). The focus of Swirrl seems to be editable tables and spreadsheets, which is different from the approach of Referata and Semantic MediaWiki, where individual pages (corresponding to cells in spreadsheets) are what’s edited. Swirrl more closely resembles SocialCalc, the application formerly known as WikiCalc, which is a wiki-spreadsheet application, though it’s not hosted, and not semantic; and Dabble DB, which also offers hosting, plus rather sophisticated editing and viewing of spreadsheet data, but without the semantic-web or wiki functionality.

Do I view any of these companies or applications as a threat to Referata? Not at all. The big hurdle for Referata and the SMW-based extensions has always been lack of awareness about structured/semantic wikis and their possibilities, as opposed to any specific competing solution. The number of people who could potentially benefit from semantic wikis but have never heard of them will, for at least the next few years, vastly overshadow the number of people who have heard of more than one structured/semantic wiki and have to choose among them. It would be great if, when describing my site, I could say “it’s a lot like Swirrl”, as opposed to having to explain semantic wikis (and in many cases, wikis themselves) every time. So I’m rooting for them to become a household name.

I should note that there are certain big advantages in being able to edit data in table form, as opposed to page by page, and I’m hoping that the SMW-based system will be able to handle tables before too long. There are various components needed to get that fully working, but one interesting bit of news is that the latest version of Yahoo!’s Javascript library, YUI 2.6.0, released last week, includes a very nice editable table input that I expect will be what Semantic Forms uses for that purpose.

With the downfall of three major banks recently (Bear Stearns, and now Lehman Brothers and Merrill Lynch), we may be seeing the death of “Wall Street” - i.e., the the end of New York as the financial capital of the world. Nicole Gelinas writes that it could have a disastrous impact on New York City as a whole, given that financial firms currently represent about a third of the city’s economy. I guess we’ll see.

Author David Foster Wallace’s suicide last week probably would have been much bigger news had it happened ten years ago. His novel “Infinite Jest” is still one of my favorite books, and, in my opinion, may be the Great American Novel, if such a thing has ever been written.

Everyone’s talking about the U.S. presidential campaign, but no one has mentioned the that the Obama-Biden ticket is noteworthy for being (I believe) the first to have a name composed of alternating vowels and consonants.

I really like Google Chrome, Google’s new browser - it might be the fastest browser I’ve ever used.

I’m also a fan of the Samsung Instinct, the iPhone clone for non-AT&T users - I just got one last week, and emailing and texting are a breeze.

The Technology Review has their annual “Young Innovators Under 35″ issue, and once again I’m not on the list. But another article in the same issue (registration required) mentions a site, SNPedia, that uses my software. So that’s nice.

I put together a “quick reference” guide, also known as a “cheat sheet”, for Semantiic MediaWiki, that covers the syntax for SMW plus three of its extensions: Semantic Forms, Semantic Drilldown and Semantic Calendar. It’s available in both PNG and PDF formats, and ready for printing in each one; you can see it here:

SMW quick reference.png

SMW quick reference.pdf

Hopefully this will be a useful reference; it’s the kind of thing that I think would have helped me a lot when I was first starting to use SMW.

The guide was created in Inkscape. The visual format was heavily inspired by the cheat sheets at Added Bytes (the site formerly known as ilovejackdaniels.com); really an exemplar in the concise display of data, as far as I’m concerned.

Just to show that you never know where ideas will come from, the idea for this reference actually came to me in a dream about two months ago; though in my dream the SMW reference guide looked like one of those laminated, back-and-front-of-the-page study guides you sometimes see in bookstores, instead of just a one-page sheet. Also, unlike in my dream, P. Diddy probably won’t show up to thank me for having created it (no, really).

I went on a trip two days ago to meet the “Tetherless World Constellation”, a research group at Rensellaer Polytechnic Institute, a university in Troy, New York. The group is headed by James Hendler, who is a fixture in the semantic-web world, possibly best known as one of the co-authors, with Tim Berners-Lee, of “The Semantic Web”, the 2001 article that introduced the idea of a semantic web to many people (as could be expected, he’s done many other things). To my surprise, the group has embraced Semantic MediaWiki, Semantic Forms and some of the other extensions in a big way in the last five months or so - they even run their own pretty extensive semantic wiki. It certainly felt a little surreal, meeting various people who work with my code and use it for their research. They’ve already created some very interesting extensions of their own, as well as useful additions to my own extensions that I hope to be able to incorporate into the code soon. Prof. Hendler has interests that extend in many directions, and he knows a lot of people, so it was interesting, in my talk with him, to hear his conceptions of how my projects fit into the overall semantic world. He also told me about some research collaborations they’re doing with outside groups, that were nice to hear about.

I got back four days ago from Wikimania in Alexandria; I haven’t had that much free time since then, between resting a lot, and trying to take care of a bunch of issues that come up during the week I was gone, and entertaining some people. Anyway, the conference was great, and Egypt was quite interesting too, though an exhausting country to visit. I saw a good amount of Alexandria, and after the conference I spent a day in Cairo and right-nearby Giza with some other people from the conference (that would be “Wikimaniacs”), where we saw some pyramids and the Sphinx. I have a bunch of photos, from then and from the conference itself, that I hope to upload soon.

As for the conference, I met some very interesting people, saw some amazing talks, and had a lot of nice conversations about all manner of wiki-things. I sent an email to the Semantic Forms mailing list summarizing the technical aspects of the conference, which you can read here.

My own presentation went fine, I thought. All presentations were videotaped, though mine, like the vast majority of them, still isn’t online, which makes me fear that it might not be available for a good while, if ever. Anyway, I uploaded the slide show that I used for the presentation;
here it is, in PDF form.

I’m very pleased to announce the release of Referata, my Semantic MediaWiki-hosting site. Wikis on this site have usage of not just SMW but Semantic Forms, Semantic Drilldown, Semantic Calendar, Semantic Google Maps, Widgets and a variety of other helpful extensions that aren’t yet available on any other wiki hosting sites. This site thus is a technical, and also, I believe, a user-interface, pioneer, in that I think it’s the first site that lets users create a site with a well-supported data structure, that’s also world-editable. It’s also very easy to set up - someone who knows what they’re doing could create an entire site, with a set of interconnecting data types, in a few hours; trying to create that same set of functionality from scratch, using a web programming language like PHP or Ruby on Rails, could easily take a few months.

I’ve been working on this site for a long time (over a year, in one way or another), and I’m happy with the way it’s turned out. Basic usage of Referata is free; you can sign up and, within five minutes, have your own wiki that can become a collaborative database. There are two service levels that require payment: “Premium”, which costs $20 a month, lets you make your wiki private, so only members can read it, and “Enterprise”, which costs $250 a month and is geared to businesses, lets you set the look (”skin”) of the wiki and use a non-Referata domain as the site’s URL, among other bonus features.

As a side note, I’ve remarked before (in real life, not on this blog) that I don’t think that Semantic MediaWiki and the related extensions have gotten enough attention so far from the outside world, given what I consider their importance; they’ve been essentially ignored by the mainstream press, VCs (not that I’ve tried to contact any, but still), tech blogs and even, for the most part, the semantic-web-focused sites and blogs. One theory I have to explain is just the lack of a coherent name to refer to all of it: people need some kind of label to give to an entire set of functionality, and currently there’s no such thing. Even the name “Semantic MediaWiki” by itself is rather unwieldy, and that’s just one extension: it doesn’t include the main application (MediaWiki), plus the six or so other extensions that can be used in conjunction with it to create a fully structed site. Perhaps what has been needed is a focal point, with a clear name, that provides something tangible for people to look at; and maybe Referata is, as they say, the right candidate for the job.

If you’re curious about the site, feel free to create a wiki of your own; or you can just test things at the Scratchpad wiki, located at scratchpad.referata.com.

The latest feature added to Semantic Forms is file uploads, added yesterday. This is a big improvement to the technology, because file uploads are a standard part of real-world forms, and one of the last things that SF couldn’t duplicate yet; before, forms had to tell users to upload their files separately, then come back to the form when they were done. The way it works now is fairly ingenious, and was basically conceived by the developers of Chickipedia for use on their site. They were nice enough to send me their code, which meant that I didn’t have to do much work to add the feature. The real heart of this functionality is the fantastic Javascript library Floatbox, which is similar to the better-known Lightbox but allows for popping up full HTML pages, and doesn’t rely on external Javascript libraries.

Speaking of Chickipedia, it’s mentioned in the current issue of Wired in a tongue-in-cheek sidebar on the “8 Best Non-Wikipedia Pedias”. I was going to say that this is the first time a project I’m indirectly involved with was mentioned in Wired, but then I remembered that they had an insert about the interactive wine bar at the Adour restaurant a few months ago (that piece is not online, apparently), and I was actually even more directly involved with that project. So, it’s the second time. It still feels good.

I gave a talk two weeks ago (I really need to update more frequently), with Sergey Chernyshev at the New York Semantic Web meetup, on the subject of Semantic MediaWiki, Semantic Forms, and some of the associated extensions (what I sometimes try to call the “MediaWiki semantic suite”). The audience was about 25-30 people, and I thought it went well - I got the sense that people understood the basic philosophy of structured semantic wikis by the end. In a bit of self-reference, you can also see the page for this presentation on Sergey’s wiki, techpresentations.org.
They see you always learn about the subject matter when you explain it to others, and for me this was no exception, perhaps surprisingly since it’s been over a year of working on the project. One thing I brought up during the talk, that I hadn’t fully thought of before I started preparing for the talk, was how a semantic representation makes creating generic software solutions for data very easy. One big thing in web programming lately has been frameworks - Ruby on Rails, most notably, but also Symfony for PHP, etc. These frameworks all make web development easier by looking at the structure of the tables in one’s database, and making all sorts of assumptions about how that data will be used - if you have a table called “Cars” with various fields, chances are that the application that uses it will need a class called “Car”, holding those same fields, and will need some web page to let someone add a car by filling in those fields, another one to display a single car and all those fields, another one to delete a car, etc. So the framework does this work for the programmer in advance, eliminating the need for a lot of low-level hacking. Well, extensions like Semantic Forms and Semantic Drilldown work in the same way, though their job is significantly easier because, instead of having to deal with numerous tables, with numerous fields in each one, there’s essentially only one database table, with just three fields, holding the full store of both data and meta-data (e.g. the type of each field) in semantic-triple form: no need to deal with all the complexities that a database structure can possibly have. Similarly, one can easily change the “data structure” of a semantic data set without needing to do any re-coding.

Semantic representation is usually described as useful because it lets you easily share data, but I think its flexibility as a data source for generic applications might be equally important.

Two notable sites have popped up recently that use my Semantic Forms extension to enable collaborative editing of data, pretty much on opposite ends of the spectrum.

First, the FEMA Region III Levee Accreditation Knowledgebase, which is currently being assessed by various people at FEMA’s “Region III”, which is essentially the mid-Atlantic states of the United States, as a possible system for recording the readiness of levees. Post-Hurricane Katrina, the state of the nation’s levees has obviously become much more portentous, and the fact that my technology could become part of the solution is thrilling.

The other site is the delightfully crassly-named Chickipedia, a wiki for information about attractive female celebrities. Launched by Break.com about two weeks ago, this is by far the highest-profile site so far to use Semantic Forms. The interesting thing is that, of all the many write-ups the site has gotten online, none have indicated awareness of the semantic technology that powers it, or the fact that, unlike Wikipedia (which it deliberately compares itself to) it provides forms for entering content; instead the focus has been, perhaps understandably, on the, er, ample content. On the whole, that’s fine by me, since my goal in the extensions I’ve created has always been to make the underlying semantic technology as invisible as possible. It’s true that it would be nice to get more recognition of the technology, although I suppose that’s what I’m helping to do now.

So, those are the two big new sites; I like the Apollonian/Dionysian, James Bond-esque duality of it.