• all code for the blog is now in svn.
• made updates to wordpress much easier
• using a lighter theme, which is nicer for code snippets

I’m still annoyed by how infrequently RSS widgets update, solution forthcoming.

Programming, especially with databases, used to be challenging. I remember having late night meetings about tables, normalization and migration and how best to represent the data we have for each packet capture. For a startup, these kinds of late night meetings are critical in establishing a bond amongst the engineers who are just learning to work with each other. NoSQL destroys this human aspect in a number of ways. more

I was (almost) worried about Xdebug no longer being free. I think this comment from Bertrand sums it up well:

Now THAT is a smart move. OSS is definitely not a way to make money. You should also definitely strengthen support for Solaris, as it being non-free again definitely means it could use a non-opensource XDebug. more

And of course, xkcd pwnd everyone today with their hardcore nerd cred:

Happy April Fools’ Day!

Women’s contributions often go unacknowledged, their innovations seldom mentioned, their faces rarely recognised. We want you to tell the world about these unsung heroines, whatever they do. It doesn’t matter how new or old your blog is, what gender you are, what language you blog in, or what you normally blog about – everyone is invited.

A lot of developers get into technology because of strong role models (anything varying from “Einstein” to “the Doc from Back to the Future” to “astronauts”). While there are many prominent male models in the tech field, it is less so for women. I do believe that one of the ways to involve women more in the community is starting to do so at a younger age (before college, before high school) and part, not all, of the appeal is having strong role models to reference.

You can help by blogging about a woman in technology on March 24th, 2010. All the cool kids are doing it! Ada wrote the first computer program (to calculate Bernoulli numbers). Do it for Ada!

Link to the footage: Inner Join / Outer Join

And the lyrics:

Craig: “Hey Maggie, I think this song’s going to be in A major…”
Maggie “Major?! but I just met’er!”

inner join outer join
do you really know the difference?
inner join outer join
a null could ruin your existence

it doesn’t matter which you choose
whichever database you use
oracle, mysql [and postgres!]
i’ve got a story i should tell
about inner and outer joins

CHORUS

do those phrases sound like gibberish
like bubbles from a talking fish?
or german or italian
or japanese or french or dutch
or maybe klingon?

CHORUS

focus on integrity
and your database will feel pretty
do you want an epic fail
or do you want your app to scale
and reach to Jupiter

where aliens can run queries
to make sure you have your foreign keys
have your ON DELETE CASCADE
relations come to aid!
your world is normalized

CHORUS x 2

Thanks for the reference!

Cassandra – “Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google’s BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.”

The huge appeal of Cassandra seems to be the approach to make it highly fault-tolerant. Writes never fail. Data is always available. No single point of failure. If you’re making a Twitter-like app, you should consider it.

Tokyo Cabinet – “Tokyo Cabinet uses hash algorithm to retrieve records. If a bucket array has sufficient number of elements, the time complexity of retrieval is “O(1)”. That is, time required for retrieving a record is constant, regardless of the scale of a database. It is also the same about storing and deleting. Collision of hash values is managed by separate chaining. Data structure of the chains is binary search tree. Even if a bucket array has unusually scarce elements, the time complexity of retrieval is “O(log n)”.”

Tokyo Cabinet is slightly newer and is apparently stupidly fast, faster than any other storage solutions out there (at least for now). It’s written in C and provided as API of C, Perl, Ruby, Java, and Lua.

Do you know anyone who has used these two already? Care to share your experiences?

The panel topic was loosely defined and the conversation tended gravitate toward Twitter and Twitter-like applications. Why Twitter? It gained huge popularity, resulting in performance problems, often leading to the surfacing of the beloved Fail Whale. It also attracts some of the biggest buzzwords of Web 2.0: social networks, information architecture, folksonomy (hash tag anyone?). In the end, Twitter has piles and piles of data – a lot of it is noise, but there is a method to the madness. The panel talked about the possibilities other applications like it mean to the Internet and to future people-oriented business.

Among others, I shared the panel with Jonathan Bryce, the co-founder of Mosso.com, now rebranded to The Rackspace Cloud and Robert Scoble, a Twitterer and FriendFeeder Extraordinaire!

The Rackspace team took a video of the panel, so check it out! (Warning: pretty big .mov file.) I already <3'ed Rackspace, as many of you know, but just in case, big props to Adrianna Bustamante and her team for the efficient nerd-wrangling! (Although I still got to mention robot overlords in the panel!)

Oh, and as for the question: “How do we make money from Twitter?” The answer is “Nobody knows, and even if they did, they sure aren’t going to tell you for free!”

The ORMs I mention:

• Propel
• Doctrine
• dOrm
• Zend Framework – specifically the proposed Zend_Db_Mapper
• Xyster Framework
• CodeIgniter
• ez components’s Persistent Object and Persistent Mapping
• symfony
• PHPLinq
• Solar’s Solar_Sql_Model
• Outlet PHP ORM
• CoughPHP
• ORMer

I plan on talking about each of these ORMs in detail in separate blog posts, so stay tuned!

At work, I design a lot of database systems that manage a lot of data. Most of these systems are put in front of real human beings who are expected to find meaningful data in a big big pile of it. The two main approaches are to use either a harsh, editorial-driven, curated system such as a category hierarchy (Rock falls under Music falls under Entertainment) or have a completely free-flowing, user-generated system such as tagging or description search. But in either case, there’s always something missing – you pick tagging, you wish people didn’t tag things with “boobies” so much. You pick a strict category structure and it just feels too restrictive. So what can you do? The March/April 2009 issue of IEEE Intelligent Systems Magazine has an article Unreasonable Effectiveness of Data.

We should stop acting as if our goal is to author extremely elegant theories, and instead embrace complexity and make use of the best ally we have: the unreasonable effectiveness of data.

The article broke my brain a little bit, but go read it, it’s interesting nevertheless.

While we’re talking about representations of data, go read about the Semantic Web – how can we tell computers and teh internets what we humans want?

If you want a little bit lighter reading, go read Bill Bryson’s books about language, specifically Mother Tongue and Made In America. Reading anything by Bill Bryson will make you a better person (or your money back).

Once you have your data, someone will inevitably ask to tell them what’s “popular”. I’m putting it in quotes, because it means so many things to so many people. Before you answer, learn a little bit about statistics. I recommend Statistics in a Nutshell from O’Reilly. Hint: “most popular” does not always mean “has most views”.

For some real-life scenarios of statistics, misuse of statistics, problems with polling plus a nice dose of politics, read Nate Silver’s FiveThirtyEight.com blog. He’s also a partner and analyst for Baseball Prospectus – you might fight baseball boring, but boy, does it lend itself toward awesome stats gathering and mangling. Reading the two might not be immediately applicable to software developers, but it’ll put your mind in a right context when trying to get meaning out of your giant pile of data.

I will expect your book reports by Monday.

Does this mean an end to MySQL? Are all companies using MySQL will now get really harsh sales pitches to use Oracle? I love Oracle and I love MySQL – the beauty of both is competition. “MySQL is better than Oracle because…” and “Oracle is better than MySQL because…” are what drives both to get better.

Good for Oracle and Sun, but I’m curious to see what happens.

On the upside, Chris and Lig will now be coworkers.