Wednesday, May 23, 2007

Human Lessons in Software Engineering?

A Software Engineer, who's struggled a good chunk of his career with poor engineering and management teams, developing applications with Microsoft technologies, learns new technologies known to be used by the type of engineers he'd been longing to work with: Ruby on Rails development on Mac OS X, among others. After a 3-year journey, he finally finds and accepts a Director position at a company that embraces his passions.

Read the story here: Strange new worlds, and programming languages...: Good bye Microsoft; Pete has now left the building!

Despite the juicy and passionately controversial platform angle, the lesson i personally take from his story is more human, than technological. It shows me that it's at least as critically important for a passionate software engineer to thoroughly interview a prospective employer, especially a lesser-known one, as it is to interview them. Gauging a company's "mentality" and engineering practices can be difficult, at times an occult art. Add to that the reality and pressure of varying personal circumstances, posturing from interviewers throwing the big "marketplace leadership, cutting-edge blah blah" marketese speak, and one can see how easy it can be for a driven employee to step into some less than desirable work environments.

Had Peter had the benefit of his current experience earlier in his career, he might very-well have recognized poor companies, poor teams. He might for example have stayed away from any company that didn't throw engineers at his interview, grilling him on challenging and exciting problems, and soliciting questions back. This by far transcends technology and development platforms.

There are great developers, building great applications that do involve Microsoft technologies. Joel Spolsky's got a few at Fog Creek Software.

But the noise is obviously high. Great companies are few, and either hard to find or extremely hard to get into. And since nobody ever got fired for building a .Net or a Java application, there are a lot of "shops" out there, that are just that ... "shops", that crank out code, and attract "the Day Coders" Peter denounces.

But in the end, Peter's strategy strikes many of us who share his ideals as one reasonable approach among others: Exploring less common, less popular technologies exposes us to passionate communities of early adopters, out to question the status-quo, driven to research better, more efficient ways to build great software.

This strategy seems to have worked well for him.

Tuesday, May 15, 2007

Introducing IBDOM

We briefly mentioned what would become IBDOM when we wrote about working with DWR.

Tonight we're pleased to release IBDOM version 0.1 under an MIT License.

The source code is on SourceForge's svn repository.

While we've yet to release the API and Usage docs, linkage to our test page from the IBDOM Site, and a cursory look at ibdom.js should reveal plenty of useful information. Update: 05/16/2006: Initial stab at Using IBDOM is up.

What is it, you might ask?

It's a 20KB (uncompressed), narrowly-scoped JavaScript library aimed at "Wielding the Document Object Model with Ease and Standards-Compliance", and most notably ease the process of injecting JavaScript Data Objects, and Arrays of JS Objects into HTML or XML documents.

IBDOM should not only be useful when working with the Direct Web Remoting framework, but also any application where you find yourself trying to plug asynchronously-loaded data into an HTML document.

IBDOM coverage on Ajaxian.

Friday, May 11, 2007

Wikitravel: Interview with the Founders

On the heels of Wikitravel's Webby Award, founders Evan and Michele took the time to share with us the personal, human and technological adventures they continue to share with "Fifty thousand of their closest friends!".

Who Are You?

We're Evan Prodromou and Michele Ann Jenkins (Maj), the husband and wife team that founded Wikitravel.

Where are you from?

We're both originally from San Francisco, but we moved to Montreal in Canada in early 2003.

Where have you lived?

We've both lived in lots of places in the US as children -- Arizona, Texas, Hawaii, Pennsylvania, Connecticut, Ohio -- and as adults we've lived and worked in a few European cities -- Lisbon, Amsterdam, Geneva. Evan lived in Mexico City as a young teen, and Michele taught English in Nepal for a year.

How many languages do you speak?

How many do we speak, or how many do we speak _well_? Evan's got about six or seven (like Greek, Dutch, Japanese) under his belt can get into working order pretty quickly, but right now it's mostly French and English that he's good at. Maj has four or five, but she's also strongest in French and English.

Where have you traveled to?

There's a point in your travel career where you start counting where you _haven't_ travelled to. Seriously, though, we've both "done" most of Western and Central Europe, most of North America, Japan, Southeast Asia. Maj has traveled extensively in Nepal and India, and visited Morocco. Evan has been to Greece and Turkey as well as Australia.

Place we want to go: Buenos Aires, Sub-saharan Africa, New Zealand, China, Siberia. Probably our dream trip ahead of us is a circuit around the Black Sea.

How did the Wikitravel concept come to you?

We've told this story so often that it's acquired a bit of a halo of myth around it. Let's see if I can do it one more time.

We were backpacking around Thailand in the winter of 2002. On an island in the east, we arrived at a wharf and took a pickup-truck taxi called a songthaew out to where our guidebook said a great hotel would be. We had to walk the last 1/2 mile, and when we got to the hotel spot, we found an empty field -- maybe a few two-by-fours sticking in the ground, but if there had ever been a hotel here, it was long gone.

As we were hiking back to the road to try and find another guesthouse to stay at, we started talking about the experience. Sure, it was unpleasant for us that our guidebook had incorrect information. But what really made us mad was that nobody else could benefit from our experience.

We could send in an email or letter to the guidebook company, but they wouldn't send a writer out to check for another year or two. In that time, how many people would make the same mistake we made? One thousand? Ten thousand?

We could also post our experience on a forum or mailing list, but it's practically impossible to glean good information from that kind of linear medium. People ask the same questions over and over and over, and meaningful information gets lost in the noise.

Evan had had experience writing for Wikipedia the previous year, and he thought that maybe a wiki travel guide series would work. That way, those ten thousand travellers could update the guide themselves -- they wouldn't have to wait for that one writer to finally show up.

What does it take to make a webby-award-winning site?

Fifty thousand of your closest friends!

Seriously, we're very proud of the work we've done with Wikitravel, but we're also humbled by the outpouring of generosity from the Wikitravel community. We were lucky to have this idea at the time when we did; people were really ready for a project like this. Wikitravel is much bigger than we are.

If we had to think of something that we did to make Wikitravel successful, it would be that we kept our vision of the site persistently throughout its lifetime. We've changed a lot of things on Wikitravel in the last four years, but we've kept true to the core vision. It's a vision that's shared with those 50,000 people, and that's probably what's made the site so successful.

... with our spouse

That's been fun. It's not easy to work with your spouse, but we've had a deepened connection working on such a big project. We probably wouldn't know each other as well, or in the same way, if we hadn't worked on Wikitravel together.

Probably being married we're both more and less effective as a team. We tend to have the rest of our life bleed into our Wikitravel discussions ("You didn't rotate the server logs!" "Well you forgot to take the trash out!"), but we also understand each other so well that we can speak in shorthand about the site. ("Did you...?" "Twice." "Don't forget to..." "I made a cron job for it.")

What do you believe are some of the factors behind Wikitravel's exponential growth?

There are a lot of them that are working together. One of the biggest is the growth of Wikipedia. People around the world are becoming familiar with wikis and with Free Content through their knowledge of Wikipedia. That means that when people find out about a site like Wikitravel, they're usually already aware of what we're trying to do, generally.

Another is the growth of Free Content projects like Creative Commons. Wikitravel started using Creative Commons licenses pretty early, and our site has grown as the CC project has grown. They've given us a lot of support and attention, which is really appreciated.

Finally, I think people are just getting used to the idea of getting the information they need online. More and more, people are turning away from paper books, with their long editorial cycles and their ivory tower attitude, to the faster, more up-to-date, and more egalitarian mode of the Internet. Books are really, really convenient, but the way reference books are made today is broken, and the Internet is one way to fix them.

When and how did Wikitravel become your full time occupation? What were you doing as your day job before?

Well, that depends on how you define "full-time occupation". When we started Wikitravel, Michele was in grad school at McGill University, and Evan was working on his first novel. But it didn't take long before Michele was working about half-time on Wikitravel, and Evan was putting in a full 8 hour day on the site.

When our first baby (Amita June) was on the way, we knew we had to get serious about working outside the home. We'd both done a lot of Internet work when we lived in San Francisco, as programmers or as programming managers. So Evan took a job as an IT manager here in Montreal, and Maj worked as an information architect for an immigration Web site.

That was probably the toughest time, because we were working pretty demanding jobs, and then trying to fit in another full-time job --Wikitravel -- into our free time. It wasn't easy, especially since we were expecting a baby coming soon.

All this time Wikitravel was growing by leaps and bounds, and getting a lot of press (Wall Street Journal, New York Times), so our out-of-pocket costs went up, and the work we had to put into the site went up too.

So when Internet Brands approached us about Wikitravel, it was like someone throwing us a life preserver. We'd had offers to commercialize Wikitravel before, but we'd never had anyone talk to us who understood the site's mission and purpose. IB got what Wikitravel was about, supported the license, had a clear and fair plan for monetizing the site, and offered to pay us to keep making the site run.

What's your technical background?

Evan's been working as a programmer since the late 80s, when he did contract programming during college and scientific programming for research groups at Berkeley. He started working in Windows programming in 1990, and ended up working for Microsoft Corporation. In 1995 he switched to Web programming and worked on a lot of great Internet projects in San Francisco. He helped develop the security protocol SAML and then left SF when he was laid off from RSA Security. Since the mid-90s he's been an Open Source enthusiast and has been a member of a number of Open Source development teams, including Debian GNU/Linux.

Michele received an undergraduate degree in computational linguistics at UC Santa Cruz and worked on a few commercial packages before joining Salon Magazine in 1999. She helped Salon switch their content management system to a Perl-based system that eventually became the Open Source project Bricolage. She left San Francisco in 2000, travelled in South Asia, then worked for the World Health Organization in Geneva, Switzerland for 3 years. She did a master's degree in information.

Why did you pick MediaWiki?

That's a good question; we often wonder that ourselves. The main reason is probably the same as for most MediaWiki sites: it's the software that Wikipedia uses. At the time we started Wikitravel, we hadn't used a lot of other wiki software, so MediaWiki was pretty much the only thing we looked at.

At the time, MediaWiki didn't even have a name; it was just "the Wikipedia software". There were occasional stable code releases (just dated, no version numbers), but we usually ran the software that was in the CVS repository, just like Wikipedia. There weren't a lot of other MediaWiki sites around at that time; probably Memory Alpha and LQWiki were the other main ones running at the time.

What Media Wiki Hacks/Extensions have you implemented?

Probably the most important extension we've added has been a tool that allows users to define RDF data inside of pages. That's allowed our users to define structured data content, like geographical relationships, inside of tags in the Wikitravel pages, without us having to re-write the database schema to support it.

We've built a couple of cool extensions that use that RDF data. For example, our Breadcrumbs extension builds a hierarchical list of geographical areas -- cities inside regions inside countries inside continents. We use the geodata in the RDF tags to make maps, using a great mapping toolkit called Mapstraction. We link loosely-related pages to each other using RDF, and we use RDF to define "Docents" -- people who take responsibility for certain guides or destinations.

Besides that, we've tried to track some of the important identity protocols on the Internet, so that Wikitravellers' identity on sites beyond ours can be integrated into ours. So, we built an extension that supports the single-signon protocol OpenID, which is now in use on a number of different MediaWiki sites. We also built an extension to support MicroID, which lets users "claim" their user pages on services like ClaimID.

What are some of the tricks you've implemented for scaling Wikitravel?

In 2004 we developed a great tool for scaling Wikitravel. We call it "Cache404". It uses a feature of Apache that can call a script if a given file isn't found. We use this to build an pretty sneaky caching system that writes out HTML output to the server's file system, so that Apache can serve the plain HTML files next time the file is requested. It's really, really fast -- Apache is really good at serving flat files.

That's been the big thing that keeps Wikitravel running at such great efficiency. We're still running only one Web server and one database server! But we have to really stretch our resources to make that work. We use memcached, the memory-caching daemon developed by Danga Interactive, and we use eAccelerator, a great PHP accelerator. We try to squeeze as much speed out of PHP as we can... and we try to run PHP as seldom as possible.

What's the role of Microformats in Wikitravel?

A lot of what Wikitravellers do is put together lists of things: lists of hotels, lists of restaurants, lists of bars, lists of museums. We realized early on that having a consistent format for all these kinds of listings would make reading and finding them much easier for people. Once you knew that the address came first, then the phone number, then the hours, etc., you'd easily be able to find what you needed in each listing.

When we first heard of Microformats, we thought: hey, we're doing this already. We were in the process of formalizing our listings formats, so we took the extra step and made the HTML output conform to the hCard format. hCard is the microformat for "contact info", so people who have a microformats-enabled browser, like the Operator extension for Firefox, can save Wikitravel listings to their contact database, to help with their travel planning.

We use a few other uF's, like "geo" for our lat/long info. We're going to be tracking the microformats effort carefully; semantic output of HTML makes Wikitravel data available to a lot of people.

What will Wiktravel look like in 5 years? 10 years?

As people say in Southeast Asia, "Same same, but different." I think that what we'll see on the English Wikitravel site in five years will be a lot broader and deeper coverage. We've still got a lot of the world to cover -- we've got articles for about 13,500 places right now on our English version right now, and our best guess at an outer limit is around 100,000 articles. We also think that those articles will be full of more information than they are now.

We also hope to have more languages supported. We're at 16 languages right now, which is pretty impressive, but we still have a lot of important languages that aren't covered. I hope that five years from now we have thriving Russian, Arabic, Chinese Wikitravel versions, and dozens more. Each language that we add means more depth of information for all our language versions.

We also think that in the five year time frame, the Wikitravel Extra features will be mature and the data stored in Wikitravel Extra will provide a vital third dimension to Wikitravel proper. Wikitravel Extra is our effort to provide "opinionated" information (reviews, blogs, photos, links, social networking, trip planning) alongside the objective content in Wikitravel. Although it's in its infancy today, we think that in five years it should be a strong competitor with other personal-opinion travel sites.

Probably in five years we'll also be available on a lot more formats. We're already making plans for a Wikitravel line of printed books. Hopefully also in five years the mobile market will be stable enough, and mobile data will be cheap enough, that Wikitravel will be available on mobile devices like cell phones.

In ten years, we think that Wikitravel will be the basis of most travel information on and off the Web. That may sound presumptuous, but probably when the Open Directory was created, the idea that Yahoo!, Google, and most other Web portals would depend almost entirely on Open Directory for their listings would have sounded absurd.

What we think will stay the same will be Wikitravel's welcoming culture and its dedication to Free Content.

See Also:

Creative Commons Interview
Wikitravel FAQ

... and see what sticks.

Yesterday was a fine day.

A developer on my team prototyped a possible solution to questions and theories that had been previously raised to improve usability and increase conversion on one of our applications.

He got to run it by various stake-holders. After constructive feedback and suggestions, they concluded: "Sounds good, let's try it". The whole decision-making process took but a few minutes.

I see this scenario unfold on a daily basis across the entire engineering team, deeply engaged in working closely with Product Management and Business to promote the success of products we build.

While there are key final decision makers who "make the call" in absence of consensus, everybody here seems open to execute on fresh opinions and innovative ideas.

Wednesday, May 2, 2007

Wikitravel Wins the Webby!

Awesome news! Wikitravel won the much deserved 2007 Webby award in the Travel category.