Offermann.us

Amazon and Google are true pioneers, making cloud computing accessible to everyone. I’m fascinated with their radically different approaches, and I find I just can’t read enough about either of them!

Recently, I’ve collected so many links about Google App Engine and Amazon Web Services, that I’ve decided to share them on a linkblog that I’m calling Amazoogle News.

My goal is to post daily links that cover news, commentary, success stories, developer HOWTOs…basically, anything I find interesting that relates to either Amazon Web Services or Google App Engine.

For anyone similarly obsessed, check it out.

Five years ago, a story made the rounds of Slashdot and Ain’t It Cool News that seemed too impossible to be true. A tale of inspiration, and obsession, and pushing way past any preconceived notions of what is possible for three 12 year-old boys to accomplish.

The story: In 1983, Chris Strompolos, Eric Zala, and Jayson Lamb see “Raiders of the Lost Ark” for the first time and fall in love with the movie. So much so, that they decide to recreate the movie, with themselves in the starring roles. They star as Indiana Jones and Belloq. They direct. They create their own sets, make their own costumes, do their own special effects, and do their own stunts, all the while filming with 1980s era video equipment. (No digital editing for them.)

The boys work for seven years before they complete their project: a shot-by-shot remake of “Raiders of the Lost Ark”.

Very few people have seen this film. Due to copyright issues, it’s never been issued on DVD (and probably never will), so it’s only been screened at a few film festivals and special charity events over the years.

On Friday night, “Raiders of the Lost Ark: The Adaptation” got its Portland premiere.

It’s an amazing thing to see. Sure, as the makers admit, the video quality is not-so-great, and the sound quality is worse. Instead of Amazonian natives chasing Indiana Jones, you see blonde boys in grass hula skirts. An “Arab native” rides a bicycle instead of a motorcycle. And, they didn’t have a monkey, so a small dog fills in the part nicely. But, as you watch, none of that matters. You really are transported, just like watching the original. And, at the same time, you can’t believe how they are able to so faithfully recreate these famous scenes.

The shootout in Marion’s bar as the the bar is set ablaze? It’s in there. How about the stunt sequence on the moving truck? The one where Indiana Jones is hanging off the front of the truck, lowers himself under it, and then lashes his bullwhip to the back of the truck so he can pull himself up. Yep, that’s in there too. They even borrowed a submarine from the US Navy to film several scenes.

Don’t believe me? See for yourself. Here are side-by-side shots from the original and the remake.

After the lights came up, Chris Strompolos and Eric Zala answered some questions from the audience, where we learned:

• They had to create the giant rolling boulder five times before getting it right. The first was created in their bedroom, but when they finished, it was too big to roll out the door. One was an unimpressive three feet tall, while another one made of chicken-wire blew away in a hurricane. Working with fiberglass turned out to be the key to getting it right
• The scene where they open the Ark of the Covenant and Belloq’s face melts required a plaster mold of Eric’s head. While making it, Eric nearly suffocated, since, instead of using dental plaster, they mistakenly used industrial plaster. Eric was rushed to the hospital with his head encased in plaster, where doctors sawed and chiseled and cut it away. Eric gave up one eyebrow and all his eyelashes for that scene!
• Chris’s cabin scene with Marion was not only his first kiss on screen…it was his first kiss ever.
• The boys were fearless with fire, and it’s a miracle they didn’t burn their house down, or seriously hurt themselves. Not only did they light numerous fires with rubbing alcohol in their basement, but once Eric lit himself on fire with gasoline.
• Steven Spielberg watched their version two times. When the filmmakers met Spielberg, he paid them a huge compliment: “Even I was inspired by what you accomplished.”

UPDATE: I just realized that I left out my favorite part from the Q&A.

• When they started this project, they didn’t have a copy of the movie on videotape, so to plan out their version, they created 600 individual storyboards. Mostly they were working from memory, but they also had a copy of the script and had a comic book version of the movie. These storyboards guided their work for the entire project. When the laserdisc version of “Raiders of the Lost Ark” came out a couple years later, Eric and Chris checked their work, and discovered that their storyboards were largely accurate.

Video from Eric and Chris’s Q&A are now posted on Youbube: (Part 1, Part 2, and Part 3). Also, someone else has uploaded the first 10 minutes of their remake.

Thirty minutes of playing with the Guestbook application in the Google App Engine SDK was all it took for me to get something up and running:

http://tofferapp.appspot.com

Getting started really couldn’t be much simpler.

How timely. The NY Times just happens to weigh in on the “Google Twin” phenomenon today: Names That Match Forge a Bond on the Internet.

So, I was wrong…My Aussie “Tom Offermann” friend isn’t my doppelgänger. He’s my Googlegänger!

Today’s theme is “Lock-In”. If you build a Web app with Google App Engine, can you ever pick up and leave?

This fear can certainly be overblown. As DeWitt Clinton (Google developer) points out “beyond the datastore interface, there are few parts of the stack that are all Google-specific.” He also clears up a common misconception: utilizing Google Accounts for authenticating your users is strictly optional. You won’t get locked in there unless you want to be.

However, the concern is legitimate when it comes to the database. Once you build on top of BigTable through the Datastore API, you’re not going to be able to move easily, because, right now, there’s nowhere else to go.

But, that’s right now. In the near future, I think you’ll have options, and lock-in may not be as much of a concern. At least, that’s a common thread through several interesting posts I found today:

• Cloud: commodity or proprietary?: Doug Cutting (Head of the Hadoop Project) stresses open-source as the best alternative. He thinks that we can’t count on “cloud providers” to build on top of a non-standard APIs, and that only open-source implementations can counter their inevitable proprietary APIs.
• App Engine: Commodity vs. Proprietary: Ian Bicking is a bit more optimistic. To him, Google’s APIs look “reasonably abstract and reasonably minimal” and “seem to lack the stench of proprietary APIs”.
• Will Google fulfill the promise of WinFS?: Ivan Krstić predicts that we’ll see “people begin to implement their own Google datastore API-compatible backends on top of everything from Apache CouchDB to (awkwardly-fitting) relational databases like MySQL and Postgres — which, make no mistake, will happen very soon.” Ivan also dissects the in-memory backend of the datastore that Google includes in the SDK, and concludes that this technology is not suited solely for the Web, but that it’s “march to the desktop is all but inevitable.”

“Ultimately, we are trying to provide an simpler alternative to the traditional LAMP stack.” —Kevin Gibbs, App Engine Tech Lead

Google’s announcement of their new App Engine was a bombshell. A big, big story that probably inspired more analysis and commentary than any story since Facebook opened up their Applications to all developers.

So, how did most bloggers frame this story? They described it as a Battle of the Titans. Goliath vs. Goliath. Google vs. Amazon. Look through the coverage on TechMeme and just try to find an article that doesn’t compare and contrast Google App Engine to Amazon’s Web Services (EC2, S3, etc.).

But, comparing Google App Engine to Amazon’s offerings misses the point. Google didn’t set out to build a killer of Amazon Web Services.

Google set out to build LAMP 2.0.

Google App Engine and Amazon Web Services are Nothing Alike

Amazon pioneered offering Web services in the Cloud, and they have an extensive array of individual Web services on their menu. One service for servers on demand (EC2). One for storage (S3). Another is a database-like service for querying structured data (SimpleDB). And, those are just the most prominent ones; they’ve got plenty more.

By offering so many tools, Amazon provides a great deal of flexibility and power to complete a wide variety of computing tasks. Photo storage, backup solutions, next-generation search engines, video publishing, Facebook Apps, data analysis and big-time number crunching…they’ve all been built with Amazon’s Web services. Now, to use all that power, you need a great deal of sysadmin smarts. But, if you’ve got the system administration know how, then you’re not limited to building only Web applications. You really can build just about anything.

In fact, when Amazon’s on-demand server farm was first introduced, it was best suited for offline tasks, and it wasn’t all that great for building complete Web applications. You couldn’t get a static IP. Amazon didn’t offer a database service. And, if you ran MySQL on an EC2 instance, you had to be very careful about your backup strategy, because if your EC2 instance went down unexpectedly, then your database files disappeared as well. To their credit, Amazon has been constantly iterating and improving their Web services, so with the recent introduction of Elastic IPs and SimpleDB, building a Web app with their services has gotten dramatically easier.

In contrast, Google is not offering developers a la carte access to individual pieces of their infrastructure. We don’t get the ability to run MapReduce tasks on their infrastructure. They’re not offering API access to BigTable.

Instead, Google’s App Engine is designed for one thing, and one thing only: building and deploying Web apps. It may lack flexibility, but it’s a comprehensive system for building Web apps the way Google prescribes. (Given it’s “There’s only one way to do it” motto, Python is a fitting choice as the first and only language supported by App Engine.) How comprehensive is it? Here’s a quick list of things App Engine handles for you that you would have to do for yourself with Amazon’s Web Services: load balancing, deployment, versioning, and backups.

What’s So Great About LAMP?

Like the Google App Engine, the LAMP stack (Linux, Apache, MySQL, PHP) is also a complete system for building and deploying Web apps. It’s widely used because it offers a great number of advantages for developers:

• Easy to code. Novices can build something and get it up and running very quickly with PHP and MySQL.
• Easy to deploy. Since PHP is a standard apache module, it’s easy to deploy a PHP app. Once you’ve got MySQL running, simply upload your .php files.
• Develop locally. It’s easy to set up LAMP on your laptop, build your app locally, then deploy on the Web.
• Cheap and ubiquitous hosting. Even the cheapest Web hosts options allow you to run PHP and MySQL.

Sounds great, right? It is…at least until Web app starts attracting significant traffic, or your Web app gets slashdotted, or suffers from the Digg Effect. Then, unfortunately, your cheap, little, ubiquitous shared Web server just falls down and dies. Just when you need it most, your Web app disappears. That’s the biggest disadvantage to using the LAMP stack.

That’s not to say that the LAMP stack can’t be scaled. Facebook and Wikipedia are just two examples that prove that it can be. But, to do so requires considerable expertise and some serious cash. You’re going to need to beef up your Web Operations team to deploy an array of load balancers, memcached servers, and Web servers. You’re going to have to manage replicated and sharded MySQL databases. You’ll need to start worrying about geopraphical redundancy. And, once you have all these systems in place, then you’ll need someone to tweak and tune your Web machine to best handle the constantly changing traffic patterns for your Web app.

LAMP 2.0: The LAMP That Scales Cheaply and Effortlessly

But, what if scaling could be handled automatically? What if you could still build Web apps quickly, easily, and cheaply, and have it also scale automatically? What if deployment was still easy? What if you could still develop your Web app on your laptop?

Even better, what if you get all that, and then running your Web app was free?

No doubt these are ambitious goals. But, what if Google can actually follow through on all these promises?

Well, then I’d say Google will have just created LAMP 2.0.

UPDATE: Found a couple more interesting discussions about how App Engine is a significantly different offering than Amazon’s Web Services.

Nate Westheimer thinks that App Engine has more in common with Facebook’s Platform. I disagree with that idea, but in our discussion in the comments of his post, he pushed back against my “LAMP 2.0” idea, rightly pointing out that Google’s services were proprietary and closed-source, and so not very LAMP-like at all.

With my LAMP analogy, I was really trying to stress the completeness of Google’s solution, and it’s emphasis on making building and deploying Web Apps easy for developers, but perhaps the “open vs. closed” debate distracts from my point.

For that reason, I think that Rich Skrenta has come up with an even better analogy: Google App Engine is “Web Hypercard”.

Why start a blog now? Well, here’s one reason:

For far too long, when you search for “Tom Offermann” on Google, you don’t find me. Instead, the top results for that query all belong to a real estate agent located in the “Holiday Capital of the Sunshine Coast” in Noosa, Australia.

If I no longer wish to be overshadowed by my Aussie doppelgänger, then it’s time to create my own presence on the Internet. Gotta get me some of that “Tom Offermann” Google search traffic.