Link to the footage: Inner Join / Outer Join

And the lyrics:

Craig: “Hey Maggie, I think this song’s going to be in A major…”
Maggie “Major?! but I just met’er!”

inner join outer join
do you really know the difference?
inner join outer join
a null could ruin your existence

it doesn’t matter which you choose
whichever database you use
oracle, mysql [and postgres!]
i’ve got a story i should tell
about inner and outer joins

CHORUS

do those phrases sound like gibberish
like bubbles from a talking fish?
or german or italian
or japanese or french or dutch
or maybe klingon?

CHORUS

focus on integrity
and your database will feel pretty
do you want an epic fail
or do you want your app to scale
and reach to Jupiter

where aliens can run queries
to make sure you have your foreign keys
have your ON DELETE CASCADE
relations come to aid!
your world is normalized

CHORUS x 2

Thanks for the reference!

Cassandra – “Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google’s BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.”

The huge appeal of Cassandra seems to be the approach to make it highly fault-tolerant. Writes never fail. Data is always available. No single point of failure. If you’re making a Twitter-like app, you should consider it.

Tokyo Cabinet – “Tokyo Cabinet uses hash algorithm to retrieve records. If a bucket array has sufficient number of elements, the time complexity of retrieval is “O(1)”. That is, time required for retrieving a record is constant, regardless of the scale of a database. It is also the same about storing and deleting. Collision of hash values is managed by separate chaining. Data structure of the chains is binary search tree. Even if a bucket array has unusually scarce elements, the time complexity of retrieval is “O(log n)”.”

Tokyo Cabinet is slightly newer and is apparently stupidly fast, faster than any other storage solutions out there (at least for now). It’s written in C and provided as API of C, Perl, Ruby, Java, and Lua.

Do you know anyone who has used these two already? Care to share your experiences?

The panel topic was loosely defined and the conversation tended gravitate toward Twitter and Twitter-like applications. Why Twitter? It gained huge popularity, resulting in performance problems, often leading to the surfacing of the beloved Fail Whale. It also attracts some of the biggest buzzwords of Web 2.0: social networks, information architecture, folksonomy (hash tag anyone?). In the end, Twitter has piles and piles of data – a lot of it is noise, but there is a method to the madness. The panel talked about the possibilities other applications like it mean to the Internet and to future people-oriented business.

Among others, I shared the panel with Jonathan Bryce, the co-founder of Mosso.com, now rebranded to The Rackspace Cloud and Robert Scoble, a Twitterer and FriendFeeder Extraordinaire!

The Rackspace team took a video of the panel, so check it out! (Warning: pretty big .mov file.) I already <3'ed Rackspace, as many of you know, but just in case, big props to Adrianna Bustamante and her team for the efficient nerd-wrangling! (Although I still got to mention robot overlords in the panel!)

Oh, and as for the question: “How do we make money from Twitter?” The answer is “Nobody knows, and even if they did, they sure aren’t going to tell you for free!”

The ORMs I mention:

• Propel
• Doctrine
• dOrm
• Zend Framework – specifically the proposed Zend_Db_Mapper
• Xyster Framework
• CodeIgniter
• ez components’s Persistent Object and Persistent Mapping
• symfony
• PHPLinq
• Solar’s Solar_Sql_Model
• Outlet PHP ORM
• CoughPHP
• ORMer

I plan on talking about each of these ORMs in detail in separate blog posts, so stay tuned!

At work, I design a lot of database systems that manage a lot of data. Most of these systems are put in front of real human beings who are expected to find meaningful data in a big big pile of it. The two main approaches are to use either a harsh, editorial-driven, curated system such as a category hierarchy (Rock falls under Music falls under Entertainment) or have a completely free-flowing, user-generated system such as tagging or description search. But in either case, there’s always something missing – you pick tagging, you wish people didn’t tag things with “boobies” so much. You pick a strict category structure and it just feels too restrictive. So what can you do? The March/April 2009 issue of IEEE Intelligent Systems Magazine has an article Unreasonable Effectiveness of Data.

We should stop acting as if our goal is to author extremely elegant theories, and instead embrace complexity and make use of the best ally we have: the unreasonable effectiveness of data.

The article broke my brain a little bit, but go read it, it’s interesting nevertheless.

While we’re talking about representations of data, go read about the Semantic Web – how can we tell computers and teh internets what we humans want?

If you want a little bit lighter reading, go read Bill Bryson’s books about language, specifically Mother Tongue and Made In America. Reading anything by Bill Bryson will make you a better person (or your money back).

Once you have your data, someone will inevitably ask to tell them what’s “popular”. I’m putting it in quotes, because it means so many things to so many people. Before you answer, learn a little bit about statistics. I recommend Statistics in a Nutshell from O’Reilly. Hint: “most popular” does not always mean “has most views”.

For some real-life scenarios of statistics, misuse of statistics, problems with polling plus a nice dose of politics, read Nate Silver’s FiveThirtyEight.com blog. He’s also a partner and analyst for Baseball Prospectus – you might fight baseball boring, but boy, does it lend itself toward awesome stats gathering and mangling. Reading the two might not be immediately applicable to software developers, but it’ll put your mind in a right context when trying to get meaning out of your giant pile of data.

I will expect your book reports by Monday.

Does this mean an end to MySQL? Are all companies using MySQL will now get really harsh sales pitches to use Oracle? I love Oracle and I love MySQL – the beauty of both is competition. “MySQL is better than Oracle because…” and “Oracle is better than MySQL because…” are what drives both to get better.

Good for Oracle and Sun, but I’m curious to see what happens.

On the upside, Chris and Lig will now be coworkers.

The example should be flexible enough to fit within different ORMs. Think of a scale from ActiveRecord to something really complicated like Java’s Hibernate – that’s the scale for the ORMs. It also needs to be really intuitive, so that the audience of the talk can understand it without much explanation – I want to focus on the ORMs themselves, not something else.

At this point, I’m leaning toward an example representing a user on a social network site and his friends. Something along these lines (don’t beat me up, it’s a very rough draft):

object model:

database model:

I think it’s a good idea to keep this example specific to functionality common in web applications (the alternative is kittens, really). On the other hand, I don’t want the example to be overly complex, so that the audience focuses more on the topic of the presentation rather than the details of the example.

When you come to presentations, what do you look for in examples? Are you looking for something you can basically copy and paste and use in your code? Or are you more interested in high-level theory where examples take a back seat and are there just for flavor but not really to explain?

l2db

Not bad! You can probably get through a lot with just these three…

“Premature optimization is the root of all evil.”

In my day-to-day job, I worry a lot about writing high-performance applications, especially in the way applications manage and retrieve data. There is a huge difference between designing your application to be optimized for a specific purpose and premature optimization. It’s one thing to worry whether echo or print is faster (who cares, really?) – but it’s another to design an application to tolerate some slowness in low-traffic parts but gaining ultra fast response time on the first access to the application. For example, viewing your photo album may take a while but seeing the homepage is pretty freakin’ fast!

In a recent post, Sebastian Bergmann points out that you should not micro-optimize (in reference to a PHP micro-optimization tricks post by Alex Netkachov). I agree wholeheartedly. It’s a nice list and definitely something to keep in mind, but it should be just a small component of your developer bag of tricks! Before you can optimize, you have to design well first. Proper design leads to potential for great optimization. And to design well, you have to really understand the problem you’re trying to solve. Sebastian links to slides by Ilia Alshanetky from PHP Quebec 2009 on Common Optimization Mistakes. Ilia also points out that it’s important to understand before you start fixing:

Solve the business case before optimizing the solution.

For PHP Advent, I wrote an article Optimize This!, which talks about how to optimize an existing application. The most important part to focus on is understand what the application is doing and then figuring out where you’ll have most impact. Sure, you can use echo over print (or print over echo, once again, who cares?) to get that 0.0001% performance boost, but if you’re doing 140 live SQL queries on your homepage, your application will still have 3 minute response time and with more users, the performance will continue to degrade until the application becomes unusable.

Modularizing code (aka “Object-Oriented Programming”) makes it easy to focus on problem areas in an existing application. Designing well doesn’t always mean designing for high performance. Instead, it means designing for flexibility and atomicity – you want to be able to easily change things without huge impact on the overall system. So before you go an optimize, take a step back and think about what your application is doing and what it’s supposed to do. Go write on the white board – diagrams prove very useful in figuring out flows and functionality! Go chat in #phpc channel on irc.freenode.net – the PHP community will help (with a little bit of playful making fun of, but we’ll help). Just don’t go changing all your methods to static methods unless you’re really sure it’s the right thing to do!